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ABSTRACT 

This monograph targets what is known about counseling 
outcomes in various counseling specialities while also providing a 
comprehensive overview of counseling outcomes research* The 
individual chapters were selected to focus on what leads to counselor 
efficacy, defined as the effectiveness of the counselor in bringing 
about counselor- and/or client-desired outcomes relating to the 
client. Each chapter includes a thorough review of the relevant 
literature and offers a summary of generalizations to be drawn from 
the area reviewed, as well as personal insights as to the 
implications of the review for counseling education, programs, and 
practices. Following a preface by Harold H. Hackney, the following 
chapters are included: (1) "Counselor Efficacy" (Garry R. Walz and 
Jeanne C. Bleuer); (2) "Career Development" (Rich Feller); (3) 
"Counselor Education" (Robert L. Gibson); (A) "School Counseling" 
(Nancy S. Perry); (5) "Student Development in Higher Education" 
(Cynthia S. Johnson); (6) "Marriage and Family Counseling" (Jon 
Carlson); (7) "A Review of the Counseling Outcome Research" (Tom 
Sexton); (8) "Important Considerations in Disseminating Counseling 
Outcomes Research" (Garry R. Walz); and (9) "A Summing Up" (Garry R. 
Walz and Jeanne C. Bleuer). A description of the Educational 
Resources Information Center/Counseling and Personnel Services 
Clearinghouse (ERIC/CAPS) and an annotated list of ERIC/CAPS 
publications are appended. (NB) 
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Preface 



In our yearly meeting with ERIC/CAPS, Garry Walz broached the idea 
with me of ACES and ERIC/CAPS jointly sponsoring a program on 
counseling efficacy at the ACA National Convention. I enthusiastically 
concurred. ACES has examined and assiduously utilized the counseling 
outcomes research in the preparation of counselors in the past and 
certainly should continue to do so in the future. Both our public support 
and our effectiveness as a profession depend upon our ability to utilize 
the results of outcome research in the preparation of counselors. The 
program was well attended and enthusiastically received. Clearly, this 
initiative hit a responsive chord with counselor educators. As adjudged 
by those in attendance, interest in counseling outcomes research extends 
to those who practice counseling as well. 

This monograph serves a real need in counselor education. It tai^ets 
what is known about counseling outcomes in the different counseling 
specialities as well as providing a comprehensive overview. A frequently 
neglected but highly important topic — the dissemination and utilization 
of research results — has wisely been included. In short, it offers a "jump 
start" for counselors on what counseling outcomes research says about 
our counseling effectiveness and our personal efficacy as counselors. 
Equally important, it reveals both the strengths and the gaps in our 
counseling research. For both new and experienced researchers it can 
well be a useful source for helping to identify future emphases in 
counseling research. 

I heartily recommend this monograph to all who are now counselors 
or who arc in the process of becoming counselors. It will be of 
considerable benefit in providing useful information and in fostering 
professional renewal. 

Harold Hackney 
President, ACES, 1991-1992 



Chapter 1 
Counselor Efficacy 

Garry R. Walz and Jeanne C. Bleuer 



This monograph has experienced a protracted gestatioa The cause for 
the extended period of development has little to do with any 
difficulty in the writing. On the contrary, the various contributors have 
been eager and excited about what they were writing and provided 
quality manuscripts on time. The reasons for the extended delay are 
more akin to the metaphor of a house builder who when he sees what a 
beautiful edifice is being constructed begins to add numerous additions 
and changes to the original plan to make it even more attractive. Anyone 
who has built her/his own house, or for that matter worked on a ground- 
breaking publication, will recognize the compelling need that arises to 
add :4 just one more section." We have at last conquered that feeling and 
believe that we have included all (or almost all) that needs to be in this 
monograph. 

Two things were of most ins; *ance in our decision about what to 
include. First, we wanted to focus on what leads to counselor efficacy. 
For us "counselor efficacy" is defined as the effectiveness of the 
counselor in bringing about counselor- and/or client-desired outcomes 
relating to the client. Clearly the most important outcomes are those 
desired by the client. However, there are outcomes which counselors 
have in mind which they b-Mieve are important even though they have 
little apparent value to the client, e.g., that the client has undertaken a 
thorough career exploration before arriving at a definitive career choice. 
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With this definition in mind, we are focusing on the counselor and how 
he/she perceives counseling and the nature of their counseling 
interventions. We believe any counseling process is mediated by the 
person who uses it, hence, we need to give emphasis to the constellation 
of ideas and practices characteristic of a counselor (or group of 
counselors) and the extent to which they can be described as efficacious 
in their behavior as counselors. To speak to this definition requires that 
we extrapolate beyond the counseling efficacy research results, most of 
which relate to outcomes of the counseling interview. 

It is probably accurate to say that for a large number — if not the 
majority — of counselors, their impact on clients (students) is a function 
of their relationships with them both in and out of the counseling 
interview. We would suggest that many of the functions identified as 
being important to achieving counseling outcomes in the interview are 
also of utility in counselor/client interactions that occur outside of the 
interview. The situation in which a counselor functions may severely 
restrict the number of formal counseling sessions he/she may have while 
offering many opportunities for "counseling on the hoof or other 
counseling interview derivatives. In situations such as school, career, 
crisis, or abuse counseling the informal counseling contacts may have 
precedence over the formal contact In fact, in at least one area — school 
counseling — the advent of the rapid acceptance of the comprehensive K- 
12 school counseling program model will prioritize counselor 
adaptability in achieving relevant counseling client outcomes through a 
variety of counselor interventions not usually described as counseling 
interviews. 

A second major influence is a belief that efficacy is a dynamic entity, 
not a static "you have it or you don't have it" one. Essential to a 
counselor's efficacy is her/his capability for professional renewal. The 
vast changes occurring in our society require that a counselor be aware 
of these changes and the impact they can have upon different sub-groups 
and individuals. Knowledgeability and versatility are key ingredients for 
the renewing counselor. He/she needs a first-hand knowledge of new 
research results as well as a high degree of flexibility in knowing when 
and how to apply the new knowledge. 

Key to the professional renewing process is a counselor who has a 
personal commitment to renewal and change as well as a systematic plan 
for insuring that she/he has access to relevant new ideas and resources. 
Put another way, efficacious counselors are always at risk of "losing it" 
unless they have a plan that provides for acquiring and using the 
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knowledge vital to being helpful to clients who are themselves 
experiencing an array of unpredictable challenges to an orderly and 
rewarding life. A colleague of ours who returned to a school guidance 
program after an absence of 20 years was greeted with the comment, 
"You will be pleased to find that things are just as you left them." It is 
interesting to speculate about how many counselors are doing things 
"...just as they did" when they left graduate school or after their first 
few years of counseling experience. 

Much as we believe that career decision making is appropriately 
viewed as the outcome of a systematic process of career exploration and 
self assessment, so too is it important for counselors to have a plan in 
mind for assessing their own growth needs and how they will acquire the 
new knowledge and skills essential to their performance as competent 
professional counselors. We have observed that, lacking a systematic 
plan for professional renewal, counselors may be unduly susceptible to 
the ploys of skillful marketers who offer quick "catch-up" workshops 
and self-study programs that promise the needed skills and knowledge in 
short order. Clearly professional renewal is not something that can be 
acquired in a few easy lessons. Efficacious counselors are ones who 
have given thought to what they desire to do as counselors and have a 
plan in mind for how they can use a wide variety of resources to 
accomplish their goals. 

With our objective of advancing the concept of counselor efficacy, 
we offer here five papers written by knowledgeable and experienced 
professional counselors who have reviewed the counseling outcomes 
literature in five important areas: 

Career Development — Rich Feller 

Marriage and Family Counseling— -Jon Carlson 

College Student Services and Development — Cynthia Johnson 

School Counseling — Nancy Perry 

Counselor Education — Robert Gibson 

Each of the authors reviewed the literature relating to counseling 
outcomes and counseling efficacy and summarized what she/he believes 
are important generalizations to be drawn regarding the outcomes of 
counseling in the area being reviewed. They also offer their personal 
insights as to the implications growing out of their reviews for counselor 
education and counseling programs and practices. We find it interesting 
to note the varying degrees of optimism reflected by the authors. It also 
is apparent that the amount of counseling outcomes research varies 
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considerably by topic and area. Even in areas where there is a history of 
outcome research, one may adjudge there to be less than one had 
expected or hoped for. 

In Chapter 6, Tom Sexton provides a comprehensive review of 
counseling outcomes research. He offers a number of trenchant 
generalizations regarding what can be said about the efficacy of 
counseling. He also suggests important implications of the research for 
counselor education and supervision. When combined with the ideas and 
conclusions presented in the five "counseling area reviews" the reader 
will find much to ponder and be stimulated by. 

Chapter 7 probes the dissemination process — the means by which we 
put information into a counselor's hands. A plea is made for regarding 
dissemination as an interactive process between researcher and user with 
mutual prioritizing of goals and reporting of outcomes. Ten specific 
suggestions are also made for enhancing the quality and utility of the 
dissemination of counseling outcomes research. 

The final chapter synthesizes and summarizes the results previously 
presented in the monograph. The chapter concludes with a series of 
knowledge generalizations based on the counseling outcomes research. 
A number of these generalizations have a futuristic bent to them 
suggesting what can and should be done to maximize the quality and 
utilization of our counseling outcomes research, now and in the future. 

In this monograph there is no presumption, either explicit or implicit, 
that it contains the last word on counseling outcomes. More 
appropriately, it is intended as a stimulus for more and improved 
counseling outcomes research. And especially it is a strong voice for 
greater attention to how we communicate research results to counselors. 
Unless we can present counselors with information in a way that 
facilitates the adoption of new ideas, counseling may be fixated on 
"...the way it was 20 or more years ago." 
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Chapter 2 
Career Development 

A Baseline of Career Counseling 
Outcomes Research 

Rich Feller 



ew practitioners or researchers question the effectiveness of career 



A counseling or related career interventions in the broad sense. Yet the 
quality and comprehensiveness of the research deserves greater scrutiny 
in light of the field's broad scope and lack of consensus about outcome 
measures. Herr and Cramer's (1992) interpretation of research results in 
career education, guidance, and counseling in the aggregate are quite 
positive, and there is "no longer a major question about the ability of 
career guidance or career counseling to improve or change career 
behavior" (p. 682). 

Does career counseling make a difference and are career inter- 
ventions an answer to an expanding number of contemporary problems? 
If so, why are many career centers filled with excellent resources but 
few clients? Why does a Gallup/NCDA (1987) poll illustrate the need 
for expansion of career information, programs, and career interventions? 
Why do policy makers, state departments, educational institutions and 
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third-party payers hesitate to increase support for career counseling 
programs and interventions? While Fretz (1981), Oliver (1979) t Osipow 
(1982) and others have examined issues related to career counseling out- 
comes, this paper offers a baseline about the status of career counseling 
outcome research. From this, practitioners may ask better questions 
about their practices while appreciating the need for quality program 
evaluation. Researchers and counselor educators may gain greater focus 
in framing research questions as they add to the literature, often devoid 
of the answers sought by clients and policymakers. 

By focusing attention on career counseling, it may become a more 
potent answer to questions related to individual choices, the work woiid, 
and economic well being. Following the approach of the ACES and 
ERIC/ CAPS initiative, this paper incorporates: 

• a brief description of career counseling outcomes with reference 
to where career interventions are provided; 

• a succinct analysis of the quantity and quality of career 
counseling outcome research; 

• a discussion of major significant research findings; 

• identification of notable trends or developments in career 
outcomes research; 

• needed areas of focus for future research; 

• implications for counselor education and counseling practice; and 

• an assessment of the relationship among outcome research, 
counselor education, and career counseling practice. 

Career Counseling Anticipates a Large Reunion 

Delivery of career counseling has traditionally been perceived as a "test 
and tell" process for a small percentage of the population who could 
afford it or were connected to some educational institution. Presently, 
career counseling needs are provided within schools, work sites, com- 
munity-based organizations and private practices. Individual counseling, 
group counseling, career guidance classes, life-work planning work- 
shops, self-directed instruction, and computer and video technology 
processes are common forms of intervention. Political and economic 
factors shape the content and purposes of career counseling, therefore, 
differences in who receives and provides which services within which 
settings are great. 
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Through the early 1980s much of the effort related to vocational 
behavior found psychologists devoted to the measurement of interests, 
and the analysis of career patterns and development rather than assessing 
the effects of practice. Interest in practicing career counseling among 
counseling psychologists seemed limited and the clinical literature 
related to career counseling doimant (Spokane, 1991). 

Yet career technicians joined career development specialists, 
educational counselors and an expanding number of human resource 
specialists in delivering programs within schools, community-based 
organizations, government, and private industry. The mushrooming force 
of paraprofessional career facilitators on campuses, within JTPA pro- 
grams, agencies such as the U.S. Postal Service, and union halls 
document an expanded interest in career counseling for diversified 
populations. While for some, it is heresy to mention the value of services 
provided by outplacement specialists and a much maligned group 
referred to as headhunters or "flesh peddlers," there is a growing cadre 
of career counselors in private practice with and without a license. 
Clearly, there is an expanded base from which career counseling out- 
comes are provided. 

Career education and life span development professionals have long 
suggested that the outcomes of career interventions merited more 
attention, research and status in the helping professions hierarchy. 
Comprehensive guidance and counseling models promoted program 
evaluation and career development competency attainment. Career 
guidance, historically available for only some adolescents, gained 
greater appeal to learners of all ages. Skill, learning system and regional 
economic mismatches called for career competencies for all over the life 
span (Feller, 1991). Career Counseling: An Old Friend in Need (Dom & 
Schroer, 1983) suggested career counseling needed to return to its 
rightful stature in the counseling profession. National studies (Chapman 
& Katz, 1981, 1982, 1983; NCDA,1988; NOICC, 1988); professional 
certification efforts promoting Nationally Certified Career Counselors 
(NCCC's); and standards such as ACSCI's for CID's, and NCDA's for 
career information reaffirmed the need for quality interventions as career 
counseling embraced increased demand, respectability and account- 
ability. 
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From Career Theory to Practice and Outcomes 

A succinct analysis of the quantity and quality of research in career 
counseling inevitably is limited by the disciplinary lens with which one 
views the literature. While discreet definitions (Spokane, in press; 
Rounds & Tinsley, 1984) between career "pure" counseling and career 
interventions have been made, one must acknowledge the considerable 
overlap between many of the outcomes of career counseling and an 
expanding array of career interventions. 

There are at least three major outcomes of career counseling — 
making a choice, acquiring decision making skills, and enhancing 
general adjustment (Crites, 1981). Herein lies much of the interest in 
outcomes research. When and under what circumstances should the 
focus be on one outcome rather than another? How does one isolate and 
fully describe the treatment variable? Observation suggests that coun- 
selors help clients process information related to each of the three out- 
comes through various approaches. This is done through traditional one- 
to-one career counseling as well as through interventions ranging from 
psychoeducational groups, to computer assisted career guidance, to 
bibliotherapy. 

Research on career interventions is a relatively new phenomenon 
(Rounds & Tinsley, 1984) with little specific research conducted on 
career counseling in its pure form (Dom, 1990). While Samler (1953) 
and Thompson (1960) earlier wrote about career counseling, research 
has focused on empirical studies of career theory rather than on counsel- 
ing process or outcome (Spokane, 1991). Early studies on career coun- 
seling generated an impression that the inclusion of a career concern was 
more for the purpose of examining a specific social influence principle 
rather then for the purpose of examining career counseling (Dorn, 1990). 

Hundreds of career interventions have been introduced to effect 
change in vocational behavior. Many are reviewed in the annual litera- 
ture reviews of career development appearing in the Journal of 
Vocational Behavior and the Career Development Quarterly. However, 
Rounds and Tinsley (1984) suggest that these reviews, along with other 
reviews of the career intervention literature (Baker & Popowicz, 1983; 
Fretz,1981; Holland, Magoon, & Spokane, 1981; Krumboltz, Becker- 
Haven, & Burnett, 1979; Lunneborg, 1983; Myers, 1971, 1986; Spokane 
& Oliver, 1983; Osipow, 1982, 1987; Pickering & Vacc, 1984; Rounds 
& Tinsley, 1984; Super & Hall, 1978; Watts & Kidd, 1978) rarely 
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address how specific interventions affect measured outcomes, or base 
their interventions on a systematic classification of vocational problems. 

While most career counseling research is dominated by the study of 
career theory, use of assessment tools, and product and program evalua- 
tion, career counseling practice is not primarily research based. Develop- 
mental psychology's theoretical and empirical advances in the last three 
decades (Jepsen, 1990) resists translation into what career counselors 
can readily apply in their wort However, career intervention practices 
noted within recent works (Burck & Reardon, 1984; Crites, 1981; 
Gysbers & Moore, 1987; Healy, 1982; Herr & Cramer, 1992; Raskin, 
1987; Spokane, 1991; Walsh & Osipow, 1990; Yost & CoibisMey, 1987; 
Zunker, 1986) offer career counseling practices useful to meeting client 
goals. 

The majority of quasi-experimental research has involved popula- 
tions of college students lacking no-treatment control groups or agree- 
ment in operational definitions of outcome measures. There are few 
studies of the comparative effect of different career interventions on 
specific outcomes. Thus, the difficult task of grouping observations from 
which generalizations can be made is compounded. Few case and group 
case studies, qualitative or new forms of naturalistic inquiry are reported. 
The practicing career counselor measuring group outcomes is of little 
value unless he/she speaks specifically to client goals. For example, as 
R. Reardon and J. Sampson (personal communication, 2/15/92) advise, 
if half the clients want to expand occupations and half want to limit 
options, the Self Directed Search may show no effect because it typically 
expands options. In summary, there are numerous problems with the 
quality and quantity of outcomes research in career counseling as 
practitioners are driven to improve practices. 

What Do the Major Research Findings Tell Us? 

Career choice and development theories generated impressive amounts 
of empirical data and influence on research and counseling practice over 
the last 40 years. Osipow's (1990) theory convergence suggests a 
relatively common base of concepts upon which to improve practice 
which supports Savickas' (1989) notion that "we need to determine 
which interventions work with whom and under what circumstances" 
(p. 107). Fortunately, such documentation has the potential of directing 
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future policy, assisting in procuring funding and shaping professional 
training. 

Myers's (1971) examination of vocational intervention literature 
concluded that "one thing the studies of counseling outcomes provide to 
the practitioner is an array of treatment procedures of tentatively 
established worth" (p. 886). Fretz's (1981) review of career intervention 
effectiveness supported Takai and Holland's (1979) view that the evalua- 
tion literature of career interventions suggests that myriad, diverse inter- 
ventions lead to small detectable gains despite differences in methods, 
format, personnel and costs. 

In Career Intervention, Spokane (1991) suggests that evaluation of 
career interventions provides information needed to assess the effect of 
stated intervention goals and objectives. His distillation of literature 
reviews (1991, pp. 213-214) offers a very thoughtful and helpful set of 
observations. In abridged form they include: 

1. The results of nearly 100 controlled empirical studies with a 
variety of clients using a full range of career interventions 
demonstrates moderate but robust beneficial outcomes. 

2. Longer (at least ten sessions), more comprehensive interventions 
have roughly twice the beneficial effects of briefer interventions. 

3. Long-term effects of career interventions have been demon- 
strated for up to 6 months after treatment with little loss of 
potency. 

4. Individual counseling is the most efficient intervention in terms 
of amount of gain per hour of effort. 

5. Inventories and self-assessment devices are still the most sought- 
after and effective intervention strategies with computer-assisted 
interpretations enhancing their effects with written information 
and media presentations. 

6. Job-seeking, self-presentation skills can be effectively learned 
using role plays, role models, and group discussions. 

7. One can expect an average of one standard deviation of gain 
following a career intervention on measures of career maturity, 
decisional status, appropriateness of choice, and information- 
seeking behavior. 

8. On the average, clients will seriously consider one additional 
career option following a career intervention. 
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9. Teaching cognitively-oriented decision strategies to intuitive 
deciders is the only intervention found to produce consistently 
ineffective and/or harmful outcomes. 

10. Low self-esteem and social skills individuals, with high 
indecision and low vocational identity, or those with unrealistic 
aspirations will have better outcomes in individual counseling 
than in self-guided or group interventions. 

11. Minority and special groups will have better outcomes with 
more structured as opposed to more vague and diffuse career 
interventions. 

Crites (1987) in reviewing evaluations of career guidance program 
and intervention studies (Campbell, 1968; Myers, 1971; Oliver, 1979; 
Fretz, 1981; Holland, Magoon, & Spokane, 1981; Spokane & Oliver, 
1983; Campbell, Cornell, Boyle, & Bhaerman, 1983; and Hotchkiss & 
Dorsten, 1985) reinforced the importance of systematic activities which 
engage all students in career guidance interventions. 

Within Foundations for Policy in Guidance and Counseling (Herr & 
Pinson, 1982), Herr overviews two decades of empirical studies that 
address the effectiveness of guidance, counseling, psychotherapy and 
related processes in changing certain types of behavior. He found that 
many interventions had been combined into programs of career educa- 
tion and career guidance and that studies of cost benefit analysis in 
psychotherapy do have significance for guidance and counseling as 
reference points for outcomes likely to be true for career counseling. 

In the midst of a decade of school reform efforts, Drier (1989) argued 
that career development interventions had become a key instrument in 
developing achieving students, satisfied citizens and a productive work- 
force. His review highlights the work of Campbell, Connel, Boyle and 
Bhaerman (1983), Bishop (1985), Suk and Bishop (1985), Holienback 
and Smith, (1985), and Mertens, Seitz, and Cox (1982). Bhaerman 
(1977), Herr (1978), and Hoyt (1980) found that career education in 
various forms delivered positive career outcomes. Through meta- 
analysis of 18 evaluation studies of career education Baker and 
Popowicz (1983) found moderate positive outcomes from career 
education treatments. 

Spokane and Oliver's (1983) and Oliver and Spokane's (1988) meta- 
analyses, recognized as two of the best syntheses of the career 
counseling literature, indicate clear differences in effectiveness among 
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intervention modes. The most important influence was "treatment 
intensity," defined as the number of hours and sessions of treatment 
received. Their findings recommended more attention to design and 
analysis issues, systematic attention to the development of a complete 
set of outcome measures and complete reporting of study details for 
purposes of replication, meta-analysis and actual use in practice. 

Qearly, more professional counselors and non-credentialed career 
facilitators are providing career interventions. While some provide 
services with limited insight into the effects of their efforts or little 
knowledge of recommendations from significant research findings, most 
career counselors hold interest and hope for improving their practices. 

Which Questions Drive Research Trends or Is 
It the Other Way Around? 

Readers interested in research trends are directed to annual research 
reviews appearing within the Career Development Quarterly and the 
Journal of Vocational Behavior. Authors analyze selected articles and 
identify trends which invariably include use of broader psychological 
processes and life span issues to explain career choice and transition. 
New perspectives on career counseling as a mental health intervention 
and changing demographics increasingly illuminate career problems 
unique to special populations. Assessment tools and computer-based 
interventions along with issues dear to the family, at-risk populations, 
and vocational adjustment also influence what gets researched. 
Unfortunately, research related to these areas can be reports of basic 
science research without practical applications or potential implications 
for counseling practice (Savickas, 1989). Numerous questions generated 
from practice or driven by state or national goals often fail to be reported 
or even researched. 

Fortunately, a notable trend in outcome research that will bring 
research closer to career counseling practice is the expansion of 
scientist-practitioner teams. A team approach to inquiry holds the 
potential for framing questions closer and more responsive to the 
immediate needs of clients and public debate about education, work 
force quality, and self-efficacy. 
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Future Questions 

Before suggesting which direction the focus of research might take, let 
me reflect on two items. A conversation (M. Savickas, personal com- 
munication, 1/30/92) emphasized that many productive researchers 
whose work appears in journals have little influence on practice or 
policy. Might researchers soon commit to the issue of usefulness in their 
studies and find more reward in disseminating useable knowledge rather 
than simply "getting published"? Secondly, Hcrr's (1982) research 
agenda for guidance and counseling still offers considerable direction to 
any career counseling effect questions or outcome research proposed in 
the future. The following is offered as one such speculation: 

1. What behavioral changes are required by which populations, 
under what assumptions? What educational, economic, social and 
labor market constraints limit the impact of career counseling? 

2. What are the comparative effects of different career interventions 
on the same problems? 

3. How are different career interventions and age, ability, special 
client group, race, gender, socioeconomic status related? 

4. What are the longitudinal effects of specific career interventions? 

5. What are the cost benefit effects of different interventions to the 
individual, the employer, and society? 

6. V/hat are the effects of interventions on organizational change and 
personal productivity? 

7. What language can be agreed upon so career measures can be 
standardized or classified, studies compared, and results dissemi- 
nated to a wider audience? 

To complement such research it seems imperative (Osipo*\ 1982) 
that improvements would be enhanced through development of a career 
problem taxonomy, from which interventions could be designed and 
outcomes selected. Answering these questions is not simple, as the 
context within which career problems present themselves becomes more 
complex. Even though, a sophisticated and diverse array of intervention 
techniques is available to address these problems. As Spokane and Watts 
(1990) speculate "the next generation of career intervention research will 
probably be an exciting blend of group and single case data that builds 
on the substantial research base of nearly 100 years of research, theory 
and practice" (p. 121). 
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What's Expected of Counselor Education and Practice? 

Pinkney and Jacobs (1985) suggest that counseling psychologists in 
training and counselor educators don't value career aspects of their role 
as highly as those dealing with personal counseling, depression, and 
psychotherapy. While there have been numerous calls for increased 
emphasis on career development in counselor education (Hohenshil, 
1982; Hoyt, 1990; Johnson & Johnson, 1982; Reardon & Buck, 1980), 
reality suggests that career development remains a low priority. Often it 
is taught by junior or affiliate faculty within a stand alone course. As 
students understand career to mean more than job, and counselor 
educators understand career development as a lifelong philosophy with a 
set of competencies (NOICC, 1988) rather than a course in occupational 
information or guidance programming, its value increases. A commit- 
ment to teaching the skills needed to conduct career counseling 
outcomes research and evaluation studies can be enhanced, even as 
additional topics are called for within counselor education programs. 

Career counseling within educational settings traditionally serve 
more affluent and younger clients with career choice problems. As 
changes in the workplace, family and economy create more career 
opportunities for adults, professional counselors in an expanding number 
of settings will note greater demand for counselor skills and interven- 
tions which enhance career adjustment. Practitioners committed to 
improved effectiveness and professional renewal will be eager to deter- 
mine which interventions result in behavior change. 

Until such a body of literature is available Spokane (1991) offers 
seven corrective stops to increase counselor and program effectiveness. 
In abridged form they include: 

1. Intensifying the intervention through a longer period or different 
mix of strategies and techniques. 

2. Reviewing the goals, objectives and outcome measures to see if 
they capture the essence of the intervention. 

3. Reviewing the client's needs and problems. 

4. Instituting treatment plans which include goals, assessment 
devices and recommendations for strategies and techniques. 

5. Engaging a peer consultant to improve treatment potency. 

6. Establishing an advisory board representative of the client popula- 
tion. 
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7. Contacting program dropouts, especially if dropout rate is more 
than 30%. (p. 224) 

The present interests of counselor educators and practicing coun- 
selors may not be focused on career counseling or career intervention 
research. However, as increased demands for self knowledge, educa- 
tional and occupational exploration, and career planning are presented to 
counselors-in-training, schools, agencies and private counselors, the 
merits of career interventions will increase as clients ask "is my time 
with the counselor making a difference?" 

As counselors become trained in more than <4 test and tell" approaches 
to career counseling, clients can expect to see strategies that empower 
clients to incorporate processes helping them to monitor their career 
development (Healy, 1990). Expanding forms of career assistance, a 
proliferation of materials, and techniques provided by an expanded 
variety of career "facilitators" require an increased amount of interest in 
evaluation (Holland, Magoon, & Spokane, 1981). With so much change 
anticipated in personal and institutional needs, and so many tools 
available from which to choose, preservice counselor education and 
continuing education must accept the major responsibility for improving 
the status, quality, usefulness and dissemination of outcomes research in 
the future. 



To Those Framing the Questions 

During any study of outcomes research, behavior change attributed to 
specific variables must be considered cautiously. Deficits exist in both 
reporting and in the context of the methodology. When one considers the 
scope of differing populations, time perspectives and interventions, 
along with the dynamic context of labor market constraints and socio- 
political change, the complexity of providing career counseling based on 
research is amplified. While career counseling is an answer, we should 
remain modest and honest about what we know. Researchers and evalu- 
ators must be open to useful questions and committed to disseminating 
findings in a fashion friendly to the research community, practitioners, 
and policymakers. While practice will continue to improve as a result of 
career outcome research, career counselors are wise to recognize that 
some of the more critical questions about effectiveness are being framed 
by those outside our ranks. As a result, career counselors must delineate 
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the overlap between psychotherapy and career counseling, clearly 
establish a knowledge base and standards of practice, share intervention 
methods, and define successful outcomes and interventions (Figler, 
1984). 

While the counselor education, vocational psychology and career 
counseling community continue to improve professional standards in 
career counseling, the age of accountability, consumer advocacy and 
growing economic instability requires answers to immediate problems 
and urgently-framed client questions. 

As one scans the present status of counselors-in-training and notes 
that the majority of counseling students are middle class females, coun- 
selor educators need to be concerned about training enough counselors 
willing to practice and conduct research with an expanding number of 
diverse populations. Of equal importance is the realization that the 
existing research base in career counseling seems to have little impact on 
policy formation at local, state and national levels. Convincing aigu- 
ments verifying the outcomes and effects of career interventions must be 
developed and disseminated to policymakers. Resources are needed to 
train a sufficient number of practitioners able to conduct the research 
needed to answer the questions that will be framed tomorrow. As career 
counseling is seen as the answer, may those asking the questions be 
convinced that effective career counseling needs to be available for all. 

Providing a baseline of career counseling research outcomes and 
commentary to issues critical to counselor educators and practice is only 
a beginning. May this effort foster a commitment to the proposition of 
useful research questions, effectiveness studies, and dissemination 
efforts. A field as dynamic as career counseling cannot rely on what only 
a few know during a time when an increasing number of questions are 
being asked. 
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With unique concern for the practitioner, this chapter succinctly and 
clearly combines an overview of outcome research with special 
attention to evaluating diverse interventions and multidimensional 
rating scales. 



Chapter 3 
Counselor Education 

Robert L. Gibson 



Introduction 

As we enter the 1990s counselor education programs are being 
called upon to prepare counselors for an increasing variety of set- 
tings and to provide them with a wider breadth and depth of preparation 
as well. These changes have included the expansion of entry-level 
programs to 2-year, full-time academic programs plus 600-hour clinical 
internship experiences. Reflecting this broad view of offerings, Hollis 
and Wantz (1990) reported 44 new course additions made or anticipated 
over the period 1990-1991. Some courses (e.g., multicultural counseling, 
substance abuse, and marriage and family counseling) were added to 70 
or more counselor education programs. 

Certainly the profession has made significant progress over the past 
30 years. We note that minimal preparation standards for entry into the 
profession have risen from the 32-hour master's degree, common in the 
early 1960s, to the 48-hours (minimum) currently specified by CACREP. 
Additionally, the profession has moved to further insure minimum 
competencies for entry into the profession by setting standaids for the 
selection and preparation of counselors and by the recognition of those 
qualified through certification or licensure. Currently CACREP has 
accredited 72 counselor preparation programs. School counselor 
certification existed in all 50 states by 1970 and today 27 states provide 
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legal recognition for counselors licensed through the National Board of 
Certified Counselors. 

Of course, a hallmark of any profession is the degree to which it has 
enhanced and advanced that body of knowledge which constitutes the 
core learnings and skills of the discipline. It is through research activities 
that this advancement occurs. It is this topic, research, and its application 
to counselor education, that is the focus of this paper. 

Research in our profession has been a topic of some controversy and 
concern over the years. Back in 1964 Hector noted that our professional 
literature was replete with alarm about the lack and limited nature of our 
published research. Later, Goldman (1978) in his introduction to 
Research Methods for Counselors, wrote: 

From 1969 to 1975 I was editor of the Personnel and Guidance 
Journal. I resolved from the beginning that we would publish only 
those articles that had something to say to counseling prac- 
titioners, that we were a reader's not a writer's journal. We found 
almost no research manuscripts during those years that satisfied 
that criterion; quite a few research reports were received, 
especially in the earlier years, but almost every one of them either 
was so technical that it could not be truly understood except by 
very research-sophisticated people, or was so limited in its impli- 
cations that it really had nothing to offer the practicing coun- 
selor....! came to the realization that the problem was not 
"research" as a general idea but rather the kinds of research that 
have predominated in our field. I became convinced that the kinds 
of research methods and the kinds of research studies that prevail 
in the field are largely inappropriate or inadequate for most of the 
kinds of knowledge and insight counselors require in their daily 
work. (pp. 4-5) 

Stockton and Hulse (1983) appropriately called attention to the fact 
that "Counseling is an applied discipline with an emphasis on practice; 
yet, if the profession does not assume responsibility for intellectual 
inquiry which might provide answers to basic questions concerning 
effective practice, the field cannot advance" (p. 303). In joining those 
calling for more "useable" research in our field, we would emphasize 
that research can (1) provide positive outcomes, (2) be carried out by 
even beginning practitioners within a simple framework of research 
procedures, and (3) be interesting. 
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It must be acknowledged that even today our profession does not pro- 
ject a strong research image. Contributing factors to this image may be: 

• Much research seems to ignore the common problems and every- 
day needs of practitioners. 

• Research reports are often written in a manner that limits their 
interpretation and hence their application by practitioners. 

• Research activities and resulting research reports rarely "excite 
the imagination/* 

• The research monies made available by federal and state agencies 
are being increasingly cornered by private research and develop- 
ment corporations. 

• Research is too time-consuming and has very few "rewards" for 
most practitioners. 

• The conflict in values cited in Woolsey (1986) between the under- 
lying assumptions of the traditional methods of research and 
theories of counseling. 

Brown (1989) noting the relationship between ethical standards of 
the profession, professionalism, and empirically verified practices 
suggested that 

As counselors become accepted as professionals in society, it will 
be increasingly important for them to base their practice on 
accepted standards of care, including standards that have been 
derived from empirical data, if only to protect themselves from 
intrusive and expensive malpractice suits. And this principle will 
not just apply to private practitioners. School counselors and 
others working in public agencies are just as vulnerable to mal- 
practice suits as are those working in private settings, (p. 269) 

Having noted concerns regarding research in our profession, the 
Strategic Planning Committee of ACES recently designated as one of the 
association's strategic goals, "the promoting of scholarly inquiry for the 
advancement of the counseling profession." The attainment of this goal 
gives us cause to move rapidly to develop a research agenda for our 
association. As a prerequisite to developing a research direction, we 
must ascertain (1) where we have been, and (2) what is the empirical 
evidence that has influenced and provided the foundations for our 
professional actions. 
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Research in Counselor Education 

Initially, this discussion will examine our research heritage from the 
standpoint of trainee admissions and influences on training, and will 
conclude by suggesting some data-based futures for consideration in 
counselor preparation. 

Studies examining admissions criteria for counselor education pro- 
grams have been popular over the past 30 years. No less than 26 
research-oriented articles have appeared in the counselor education and 
supervision journals alone over this period of time, reporting studies 
investigating the predictive reliability of various admissions criteria. 

Markert and Mouke (1990) reviewed much of this research in 
counselor education admissions published between 1955 and 1989. They 
noted examination of such criteria as academic aptitude as reflected in 
GPA or as assessed by exams such as the GRE, letters of recommenda- 
tion, interpersonal characteristics, using various scales or personality 
measures and personal interviews. The findings of these studies sug- 
gested serious predictive validity for each of these criteria 

Maricert and Mouke then proceeded to survey 32 counselor education 
programs to ascertain their current admissions criteria. The criteria 
reported most frequently were undergraduate GPA's (varying minimums) 
(25); GRE or MAT (18); personal statements (25); letters of recom- 
mendation (29); individual interviews (13); group interviews (8); prereq- 
uisite course work (18); and work experiences (16). 

Another recent survey by Cristiani and Basile (1991) reported 
responses from 24 counselor preparation programs. They reported that 
the criteria most frequently utilized by their sample were letters of 
recommendation (23); GPA (22); GRE or MAT (22); goal statements 
(14); work experience (14); interviews (12); personal narratives (10); 
and interpersonal skills (9). They concluded that researchers have yet to 
provide much evidence that any of these criteria, singly or combined, 
effectively predict academic success or eventual counseling effective- 
ness. Nonetheless, many counselor education programs continue to 
apply admissions criteria that cannot be empirically validated to justify 
its utilization. 

In addressing the topic of research influences on counselor education, 
we must readily acknowledge that all empirically-derived knowledge 
appropriate to our professional mission as counselors could be cited as 
research that has potential influence on how we educate those in our 
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counselor preparation programs. I will happily admit that such a for- 
midable and extensive task is not within the purview of this brief 
presentation. In addition, my colleagues in this publication have already 
addressed research in many of the major areas of counselor education. 
Thus, this discussion will focus on (1) who arc the researchers frequently 
cited to our students, and (2) what research have counselor educators 
noted as influencing them as counselor educators. 

In identifying those theoretician-researchers whose contributions are 
frequently cited to our students, 12 introductory textbooks were re- 
viewed. It was assumed that textbooks which serve to introduce begin- 
ning students to the professional field of counseling would by their 
broader coverage be more discriminatory (in a sense) than the more nar- 
rowly focused specialized texts (i.e., career, group, assessment). The 
consistency with which the works of the individual theore- 
tician/researcher were presented across all the texts reviewed with the 
number of citations were the two criteria used in identifying those most 
frequently cited. The major contributors are listed alphabetically in their 
categories (always cited, very frequently cited, and frequently cited) 
with their areas of contribution indicated in parenthesis. 

Always cited 

Rogers, C. (Person-Centered Counseling) 

Very frequently cited 



Carichoff, R. R. 
Corey, M. S. 
Ellis, A. 
Ivy, A. E. 
Krumboltz, J. 
Super, D. 
Yalom, I. D. 



(Helping Skills) 
(Group) 

(Rational-Emotive Therapy) 

(Micro-Counseling) 

(Behavioral) 

(Career Development) 

(Group) 



Frequently cited 

Bandura, A. 
Berne, E. 
Brammer, L. 
Cormier, L. S. 
Egan, S. 
Glosser, W. 



(Social Learning Theory) 
(Trans) 

(Helping Skills) 
(Interviewing Skills) 
(Helping Skills) 
(Reality Counseling Theory) 
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Holland,^. 
Perles, F. 
Williamson, E. G. 



(Career Decision Making) 
(Gestalt Therapy) 
(Directive Counseling) 



It is fair to say that most, if not all, counselor preparation programs 
cite Carl Rogers to their trainees and many will also draw upon the 
research of those very frequently or frequently cited. 

A second approach was to more narrowly examine what specific 
research has influenced how counselor educators function. A brief open- 
ended questionnaire asking for this response was sent to a sample of 
counselor educators. 

This sample, though small, seems to validate earlier unpublished 
studies by the author (unpublished because of lack of a common core of 
research). Twenty-four counselor educators reported a wide jange of 
research studies that had influenced their practice. The individual 
researchers most frequently cited were Ivey for his microcounseling 
(four times); Carkhoff for his facilitative conditions studies (three 
times); and Holland for developing a vocational identification and 
classification system (two times). Several respondents mentioned studies 
focusing on client-counselor interactions and supervision variables, but 
did not identify any specific researchers by name. A number of other 
general areas of useful research information (twelve) were singularly 
noted, plus other researchers (six) who had made an impact on the 
counselor educators' reporting. 

At this point we can conclude that while research has failed to give 
us a single set of valid criteria for admissions, this may be more con- 
sistent with a counseling philosophy of each individual as a person of 
worth within his/her own uniqueness. On the other hand, despite 
criticisms cited earlier, our profession has drawn on both a common core 
of research data and a broad range of empirical studies to provide our 
knowledge base for much of our counselor education programs. Let me, 
therefore, turn to the more important task of examining our research 
future — a future including both tremendous opportunities as well as 
tremendous challenges. 
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Counselor Education Research: The Future 

As we enter the 1990s and emerge into the next century counselor 
education has the opportunity to not only build on the progress of the 
profession in the recent decades, but to accelerate and give it direction. 
We may accomplish this through research studies that address our own 
professional issues and, equally important and largely ignored, studies 
that address our society's major social issues and the unique role we can 
play as trainers of counselors for the future. In this quest we are also 
responding to those within our ranks who are calling upon the counsel- 
ing profession to become a more unique, viable and visible profession. 

To succeed in this quest we should remind ourselves of those basic 
marketing principals (paraphrased) which suggest that: 

• what we have to offer is needed 

• our response to this need is unique (different from our com- 
petitors and better — more efficient and/or effective) 

• we can offer evidence that what we offer leads to the results 
claimed. 

This will not be easy. Conyne (1977) voiced a concern that we've 
been aware of over the years when he noted that: 

Counselor education and guidance counseling are rooted in adapt- 
ing, applying, borrowing, and adjusting what has been offered in 
other professions — a situation stylistically similar to a person 
continually stripping, refinishing, and using old chairs found in 
garage sales. While the final products may be sound and attrac- 
tive, their basic structure remains the same. Counselor education 
could easily be dubbed a Secondhand rose." 

This continual borrowing has led the profession to contribute little 
creative and original directions to the fields of education and 
human services. It is laigely because of this situation that coun- 
selor educators and guidance counselors together are finding 
themselves in an increasingly precarious employment position. 
Controllers of the purse strings now ask about counselors with 
increased frequency, "What do they do?" "How can we know 
they are actually doing it?" and, "Are they even necessary, since 
we have a school psychologist/counseling psychologist/clinical 
psychologist/etc?" I submit that these are hard questions for us to 
answer adequately and that our difficulty arises in large part from 
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the secondhand rose position in which the profession continues to 
place itself, (p. 310) 

We do, however, have the opportunity to alter this image of an imita- 
tor and borrower. We can do this in the context of the previously cited 
marketing principles by a research initiative which: 

• responds to social needs 

• re-eraphasizes our uniqueness as a helping profession (i.e., our 
emphasis on prevention and development as well as remediation; 
the only helping profession with a career development base; the 
only helping profession that consistently works with populations 
in their formative years) 

• anticipating and preparing for the future — being proactive not just 
reactive 

• involving our graduate students as active partners in meaningful 
research undertakings 

An analysis of futures data collected and shared with the strategic 
planning committees for AACD and ACES pointed to some of these 
futures with the following implications for counselor preparation. 

Future Projections and Implications for the Profession 
Future: The Workplace 

Rapid technological change will affect virtually every work setting. 
Coupled with the continued movement towards information production 
and away from goods and to a lesser extent service production, this will 
result in mismatches between needs and workers qualified to meet these 
needs. This rapid technological change will also necessitate multiple 
career changes for the majority of our work force in the coming decades. 
The average age of this work force will continue to go up as older 
workers keep working. This raises these questions: 

• Will the older workers be willing to participate in the "life-long 
learning" needed to maintain a world class workplace? 

• Will these older workers contribute to the difficulties young entry 
level workers are beginning to experience in finding jobs? 
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Complicating this economic projection is the prediction that unem- 
ployment/underemployment, especially with minorities, will continue to 
be a problem at this critical time. By the year 2000, one-third of our 
population will be "minority.** This will amplify racism, which is already 
on the rise and threatening to be a major social problem by the turn of 
the century. 

Implications/Opportunities for the Counseling Profession: The 
counseling profession has a historical interest in, and commitment to, 
career development We arc the only helping profession with expertise in 
this area. We should expand this expertise to become "economy smart" 
as well as "career smart" We should also be more active in working 
with and researching with business and industry, vocational and tech- 
nical educators, and government to bring together youth, training, and 
jobs to find answers to problems of older workers and to develop a 
world class work force. We should be the centerpiece of any legislation 
focusing on these career issues. We are not to date! 

Future: Schools 

Learning will be influenced by a rapidly expanding information base 
requiring multi-media approaches transcending the printed word (i.e., 
electronic learning). New curricular emphasis will include new techno- 
logical/vocational curriculums (similar to those already in place in 
Germany, Japan, Korea); renewed attention to the arts; focus on glob- 
alization and living and working with an international orientation, 
environmental conservation and the schools as centers for life-long 
learning. Also, a longer school day and school year may be part of the 
future in school systems in our country. 

Schools will continue to be breeding grounds for major social 
problems, i.e., substance abuse; AIDS, and other sexuality problems; 
teen violence; racial tensions; economic and educational gaps between 
the "haves" and the "have nots." Nearly all children will be "latch key" 
throughout their school years. One result: More youth "gangs." 

Implications/ Opportunities for the Counseling Profession: Preven- 
tion, wellness and development must become major goals of school 
counseling programs, and major research agendas for our profession. 
Obviously, prevention and wellness courses must become a reality in 
our counselor education programs. Minority counselors arc needed as 
both role models for minority youth and as multicultural awareness edu- 
cators in every school. While certified minority counselors are 



33 



32 



Counselor Efficacy 



preferable, this need is so critical that consideration of training minority 
para-professionals should not be overlooked if this is the only means of 
having minority adults in every school. 

Further, we cannot continue to allow the bureaucrats, politicians, and 
others to ignore the role of mental health and student wellness in pupil 
attainment and their future citizenship. Counseling approaches should be 
applied to the classroom and the total educational environment. We must 
also anticipate that schools will have to assume more responsibility for 
underparented ("latch key") youth. School counselors must move now to 
prepare meaningful programs. We must be proactive! Program develop- 
ment courses must prepare counselors seeking employment in school 
settings to develop such components for their total school program. 

Future: The Family 

Dual career couples will be the norm by the turn of the century. Also, the 
work force will be more mobile. "Roots" will be more difficult to put 
down. Further, as the American work force becomes more stressed out 
and children have less parental contact, the probability of more stress on 
families is high. 

Implications/Opportunities for the Counseling Profession: Marriage 
and family counseling must prepare for new dimensions as traditional 
family roles, patterns of living, responsibilities and relationships are 
altered. Again, researching these new variables will be vital. 

Future: The Society 

Society will continue to experience most of the major social problems 
already present at the beginning of this decade. As a result, crime and 
delinquency will continue to increase despite tough new laws and more 
jails with low income minority populations being both the most victim- 
ized and the most incarcerated. Gang violence can be a rapidly increas- 
ing problem unless appropriate preventative action is taken. 

An increase in "people abuse" also seems probable in a more stressed 
out, less stable society. AIDS will affect epidemic numbers in the 
immediate years ahead and substance abuse will continue to be a major 
social concern. 

Implications/Opportunities for the Counseling Profession: Again, the 
counseling profession must stress and enhance its prevention skills, and 
must underscore its capacity to work with populations (schools, work 
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place, community, etc.). Counselors are "caregivers." Counselors should 
be activists for caring environments in all of society's basic institutions. 
Counselors must develop programs on human sexuality, values cterfica- 
tion and development, and ecological management 

Summary 

Opportunities exist for both research and training to serve projected 
social needs of the next decade while at the same time advancing the 
profession to the pinnacle of all helping professions. Our counselor 
preparation programs must: 

• strengthen our traditional career core 

• require courses in multicultural counseling, prevention and well- 
ness, human sexuality and research 

• expand marriage and family offerings 

• develop international internships 

Finally, we need a national research agenda that involves all of coun- 
selor education, with all of our graduate students as junior partners, 
actively involved in meaningful empirical studies that will advance and 
validate our field of knowledge and establish our reputation as a profes- 
sion in its own right 
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Introduction 

School counseling traces its roots back to the turn of the century when 
the Vocational Choice Movement was started by Frank Parsons in 
Boston. This was the beginning of "scientifically" directing students to 
specific occupations which seem to meet their individual interests, 
aptitudes and skills. When vocational guidance efforts slowed down in 
the 1920s, school guidance was refocused to address personal and social 
issues of students. The Mental Health Movement brought specialists into 
the schools who became involved in diagnosis and psychotherapeutic 
intervention. The advent of Sputnik in 1957 evoked a massive infusion 
of federal money to train high school guidance counselors in the identi- 
fication and encouragement of capable students to enroll in college, 
especially in math and science programs. In the 1960s and 70s, school 
counseling became part of a constellation of student services which 
included school psychologists, social workers, and nurses. In the 1980s 
and 90s, the birth of the Developmental School Counseling Model 
changed the emphasis of school counseling from being an ancillary 
service to the concept of being a comprehensive program, based on 
developmental principles. 
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Developmental School Counseling Model 

In the Developmental School Counseling Model, which is being 
promoted by the American School Counselor Association (ASCA) as the 
preferred format, the positive aspects of the earlier approaches have been 
incorporated into a comprehensive program which includes group 
curricular experiences in which to develop life and career skills, 
individual advisement in educational and career decision-making, and 
counseling and consulting to ameliorate individual barriers to learning. 
This program approach is proactive, rather than reactive, and 
systematically reaches all students rather than just those who seek 
assistance or are referred. Comprehensive school counseling programs 
based on this model are considered an integral part of the educational 
program rather than an ancillary service. They are designed to prepare 
the whole student to live, learn, and work in the world of tomorrow. 

Prevention and intervention are served equally in the comprehensive 
developmental model. Since it is logical to introduce preventative skills 
at the earliest opportunity, elementary school counseling has been 
growing rapidly as over one-fourth of our states have mandated school 
counseling programs at that level. Middle/junior high school counseling 
programs have also increased with the realization that career and pre- 
coilege guidance must begin before the high school years. Today, over 
70,000 professional school counselors work in all levels of schools from 
pre-K to post-secondary. 

The comprehensive, developmental model has gained much favor 
across the nation among educators. It is now time to assess its effective- 
ness. Counseling has had difficulty achieving the precision attainable in 
research in the physical sciences. Because of the intangible nature of 
many of the variables present in the counseling situation, problems of 
definition of terms, criteria selection, measurement, and control are 
formidable. 

Research 

Consequently, most of the research in counseling has relied on 
correlational strategies that can elucidate relationships among variables 
but cannot specify causality (Gclso, 1979). Tne effort to identify causal 
relationships has resulted in an increasing focus of research on very 
narrow topics that have little relevance to the day-to-day activities of 
counselors (Goldman, 1977). There has been research and documen- 



ts 



45 



Assessing and Using Counseling Outcomes Research 



39 



tation of various models of school counseling programs. These range 
from the effectiveness of small group interventions for children of 
divorce to the efficacy of school counselors based on their relationship 
to their principals. However, due to the newness of the comprehensive, 
developmental model, there have not been any longitudinal studies to 
validate the effectiveness of such an approach. Since many of the 
individual elements have been researched and found to be effective, one 
might hope that the combining of these strategies might produce a 
synergistic effect. However, until such studies have been conducted, one 
can only speculate on the validation of this approach. 

Expected Outcomes. In order to consider the research needs, one 
must identify the outcomes expected as the result of a developmentally- 
based comprehensive counseling program. Generally, ten results have 
been identified as desirable outcomes (Sears, 1991). As a result of the 
counseling program, high school graduates will be able to: 

• develop effective relationships with others 

• apply their interests, skills, and aptitudes in the world of work 

• use a decision-making process when making important decisions 

• control and direct their feelings/emotions 

• resolve conflicts peaceably at home, school, or work 

• resist using alcohol and other drugs and engaging in other 
harmful forms of behavior 

• manage stress 

• utilize resources to set and reach educational goals 

• achieve realistic career goals 

• value life-long learning 

• value traits/characteristics of people who are different 

Although the first six outcomes might be considered appropriate for 
any counseling intervention, the last four are unique to the educational 
setting. 

Effective School Counseling Programs. One of the principal tasks 
of the school counselor is to help each student utilize available resources 
to make informed educational decisions. This is usually accomplished 
through a combination of group guidance, in which information and 
sources of information are conveyed to the students, and individual 
advisement in which a student's aptitudes, interests, skills and 
aspirations are analyzed in order to make the best educational decisions. 
According to the National Association of College Admission Counselors 
(NACAQ Statement on Precollege Guidance and Counseling and the 
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Role of the School Counselor (NACAC, 1990), "Of particular 
importance to student success is access to a strong precollege guidance 
and counseling program that begins early in the student's school career. 
Early planning (e.g., secondary school course selection and precollege 
enrichment programs) can insure that students pursue the most 
challenging curriculum that results in enhanced postsecondary 
educational options." The statement goes on to delineate the components 
of an effective precollege guidance and counseling program. Most of 
them would be considered generic to an effective school counseling 
program. They are: 

• A written statement of philosophy that is consistent with the 
school's overall philosophy. 

• A written comprehensive plan of action that outlines student 
needs and sets forth goals and objectives for meeting those needs. 

• An evaluation process that measures progress toward meeting 
stated goals and objectives. 

• A focus on precollege guidance and counseling that enables 
students to prepare themselves academically for a wide range of 
educational and career opportunities. 

• Differentiated staffing that includes a sufficient number of 
counselors with counseling loads that enable them to accomplish 
program goals and objectives. 

• Effective communication with a variety of constituencies, includ- 
ing students, parents, all educators, and the larger community. 

• Counselors and other educators committed to the principles of 
equal opportunity and affirmative action. 

• A supportive instructional and administrative staff who work 
cooperatively with counselors to assist students in achieving 
educational goals consistent with their aptitudes, abilities, and 
interests. 

• An environment that recognizes each person's worth in a complex 
multicultural, changing society, one that supports and encourages 
each student to reach his/her potential. 

• Respect for student privacy and the need for confidentiality of 
records. 

• Delivery of services according to ethical practices. 

• Assurances to students and parents that counseling professionals 
have been properly trained to carry out the precollege guidance 
and counseling responsibilities. 
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• Adequate facilities, resources, and equipment to accomplish the 
goals of the program. 

These are the criteria against which any school counseling program 
should be evaluated. The same guidelines apply whether a student plans 
to continue his/her education or not These criteria will utilize resources 
to set and reach educational and career goals, inculcate a desire for life- 
long learning, and teach the valuing of diversity in all people. A 
comprehensive school counseling program, developmentally based, will 
also refine and develop life skills such as establishing effective 
relationships, becoming aware of one's feelings and emotions, learning 
methods of negotiation and conflict resolution, managing stress, and 
avoiding harmful behaviors. If these are the desired outcomes of school 
counseling programs, how should they be measured? 

Program Evaluation and Counselor Evaluation* Research in 
school counseling may be divided into two elements — program 
evaluation and counselor evaluation. Program evaluation should include 
the measure of effectiveness of individual program components as well 
as the effectiveness of the program as a whole. This might be appraised 
by evaluating students' perceptions of their proficiency in achieving the 
10 outcomes listed previously. Ideally, a comparison could be made 
between a group of students who have experienced a comprehensive 
developmental counseling program over a sufficient number of years 
and a control group who have not participated in such a program. At this 
point, such a longitudinal study has not been possible. The seeds have 
been planted, though, in such places as Des Moines, Iowa, where 
elementary counselors have been placed in all 41 elementary schools 
with a consistent ratio of 1:250. All schools are working toward a 
consistent approach using the comprehensive, developmental model. 

These students will be followed as they progress through the school 
system, provided funding is continued. Until that time, the American 
School Counselor Association has attempted to help individual schools 
evaluate their programs by offering self-audits at all levels (ASCA, 
1990). These audits are based on the assumption that there are broad 
professional principles of good practice, management, and conduct. The 
topics covered are (1) the philosophy, objectives, and policies of the 
guidance/counseling department; (2) the role and function of the 
guidance/counseling department in the school; (3) the role and function 
of the school counselor, including general role and function, personal 
counseling, career guidance and counseling, college and financial aid 
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guidance and counseling, ancillary services, testing, working in a con- 
sulting capacitv, referrals, accountability, and identifying trends and 
issues; (4) specialized populations in counseling; (5) professionalism and 
the school counselor, and (6) professionalism and the field. Many states 
such as Mississippi (Mississippi, 1987) and Florida (Florida, 1979) have 
developed manuals to assess counseling programs. Others have 
depended on the local establishment of standards and measurement such 
as that done in Eugene, Oregon (Hendrickson, 1982). 

Action-oriented research of individual program components or 
activities has not been embraced by practicing school counselors despite 
encouragement and resources provided by ASCA. Such research is 
defined as being focused on immediate application, not on the 
development of theory or on general application. Its findings are to be 
evaluated in terms of local applicability, not universal validity. Its 
purpose is to improve school practices, and, at the same time, to improve 
those who try to improve the practices: to combine the research 
processes, habits of thinking, ability to work harmoniously with others 
and professional spirit. 

ASCA has produced a professional development workshop entitled, 
"Developing Practical Action-Oriented Research for Demonstrating 
School Counselor Effectiveness and Accountability" (Allen, 1991) 
which has been given to all state school counselor associations for use in 
their states. Additionally, a booklet, "Desk Guide for School Counselor 
Research: Tips and Techniques'* (Porter, 1991) was distributed to all 
state associations. Practicing school counselors were targeted as recipi- 
ents for ASCA Research Grants but the resulting participation was very 
poor. One can only guess the reasons for the resistance of practicing 
school counselors to engaging in this method of accountability. 

All school counselors should have experienced some kind of expo- 
sure to research in pre-service training. Although fear of statistics seems 
to be common to many counselors, there is some indication that 
counselors have little desire to upgrade the skills that would assist them 
in conducting research (Wilson, 1985). Sometimes there arc also institu- 
tional barriers such as obtaining permission from parents or supervisors. 
Resistance from parents and administrators can be especially strong if 
the research design calls for some kind of control group that does not 
receive the services being evaluated. 

The question of who does not receive the treatment is always difficult 
to resolve in public school research in which presumably every child is 
equally entitled to available resources. The most frequently given reason 
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is lack of time (Vacc, 1981). However, nearly every school requires 
some type of program evaluation which, unfortunately, often takes the 
form of general reactions, testimonials, and other forms of self-report 
ASCA has since enlisted the assistance of the Association for Counselor 
Education and Supervision (ACES) to encourage counselor educators to 
assist in action research in local schools as a pro bono service to the 
profession. The University of Southern Maine requires that counselor 
interns conduct some type of action research during their internship 
within schools. They have the technical assistance of their academic 
intern supervisors to help them in the design and interpretation of such 
projects. It also provides a model for the field counselor supervisor. It is 
hoped that such application will help to remove whatever stigma exists 
that keeps school counselors from using research methods in evaluating 
their programs. As Pine (1981) stated, "A single school counseling 
program is a research gold mine in terms of important questions, variety 
and richness of data, and numbers of potential researchers" (p. 498). 
Counselors who leam to conduct research will become valued members 
of their schools, capable of not only enhancing their own program but of 
making significant contributions to the entire educational community. 

Counselors are also in the unique position of using educational 
research in advising students. However, they must be aware of the rela- 
tionship between bias (racism and sexism) and past and present research 
(Campbell, 1986). The tests counselors use, the ways test results are 
interpreted, and counseling techniques are all based on educational 
research. Counselors are cautioned to remember that not all research is 
done well or accurately; societal attitudes about people of color and 
women influence research results; and news reports and research 
abstracts do not include enough information to enable one to evaluate 
the research. Bias may be more subtle now but a counselor must 
carefully analyze the conditions of the research and determine if the 
study results apply only to people who are similar to those studied or are 
overgeneralized to include others. Counselor preparation programs 
should emphasize the importance of interpreting research as well as 
objectivity in conducting accountability studies. 

Research into counselor evaluation has produced a plethora of 
personal indicators of counselor effectiveness. No matter what the 
situation is, clear communication and support for the development of 
efficient, effective, and professional definitions of the counselor's role 
must be established between the counselor and his/her supervisor 
(Stickel, 1990). One of the greatest complaints of school counselors is 
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that they are often evaluated with the same instrument that is used for 
teachers. This is due to the fact that performance appraisal is often 
linked to negotiations. The majority of school counselors operate within 
the teachers' contract and are a part of their bargaining unit. Therefore, 
evaluation is not differentiated. However, some school districts such as 
DeKalb County, Georgia have developed an instrument for collecting 
information from assistant principals, head counselors, teachers, and 
students (Tucker, 1988). Competency areas included: Relating to Others; 
Communication; Professionalism/Ethics; Planning and Implementation; 
and Demonstrating High Expectations. Although no single instrument 
can incorporate all job-related behaviors, this counselor evaluation 
instrument includes behaviors shared by most counselors. 

The Houston, Texas, Counselor Quality Assurance Program Manual 
(Mclntire, 1983) addresses the renewal and professional growth 
opportunities for school counselors. Assessment is achieved through a 
team process. The Johnson O'Connor Research Foundation (Foxley, 
1982) administered aptitude tests to a sample of guidance counselors. 
The guidance counselor sample scored significantly higher than the 
general population in Ideaphoria (a measure of verbal fluency), 
Inductive Reasoning (quickness in seeing relationships) and Pitch 
Discrimination. The high Ideaphoria and Inductive Reasoning scores 
were expected because the counselor's job appears to use these apti- 
tudes; the high scores on the music test, however, do not have a ready 
explanation. 

One study reported a possible pitfall in the evaluation of school 
counselors (Montgomery, 1979). Counselors who support students' self- 
exploration may be evaluated less favorably than counselors who make 
decisions for the students. Two theoretical approaches to school coun- 
seling are the guidance of student personnel services approach and the 
counseling psychotherapeutic approach. If credibility data are collected 
according to the criteria of the guidance orientation, but counselors 
employ a psychotherapeutic approach, the results become meaningless. 
Yet these results are often used for decisions concerning the support for 
counseling. This study again emphasizes the need for clear communi- 
cation in evaluating the school counselor. 

The research needs for the future are those that will demonstrate the 
effectiveness of school counseling programs. If the comprehensive, 
developmental approach to school counseling programs is to be sup- 
ported, longitudinal research must be conducted to assess its value. Until 
that time, we must rely on research of individual program activities, e.g., 
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children's social skills (Elliott, 1987). The preparation for such a longi- 
tudinal study would require the development of consistent standards 
among such programs. The ratio of counselors to students will have to 
be considered because counselor availability has a significant impact on 
perceptions of counselor effectiveness (Boser, Poppen, & Thompson, 
1988). Separate studies would probably have to be conducted to deter- 
mine ideal ratios because the same program may require differing 
amounts of counselor attention according to the population served. The 
stated purpose of a comprehensive school counseling program is to serve 
all students. There will never be enough counselors to reach all students 
effectively through individual contact. Therefore, group work is 
emphasized as an effective methodology to achieve program goals. 
Research needs to validate the effectiveness of group work, both small 
group counseling and large group guidance. Finally, research needs to be 
conducted concerning the resistance of practicing school counselors to 
engage in research. At a time when the survival of the counseling 
profession in the public schools may depend on the ability of its 
members to demonstrate the effectiveness of their services, the majority 
of counselors do not engage in research or in evaluation activities. It has 
been suggested that this can happen only if the goals of the guidance 
program are restructured, altering the priority of counselors 1 tasks, and 
modifying the expectations of various groups within and outside the 
school (Wilson, 1985). 

Counselor Educators and School Counseling. Counselor educators 
have a significant responsibility to the school counseling profession for 
conducting research in the schools. First, they have the motivation to 
conduct research themselves because of the publish or perish philosophy 
inherent in most universities. Longitudinal studies will have to be 
conducted by those who have access to students across the grade levels. 
This may be a central office supervisor or a counselor educator. 
Counselor educators also have a responsibility to assure that preservice 
counselors are well trained both in research methodology and practice. 
Counselor education programs must be examined to determine the 
reasons that practicing school counselors do not use their research 
knowledge after leaving the post-secondary setting. 

Last, counselor educators must help practicing school counselors to 
use their knowledge and accessibility to students to provide the needed 
evaluation and research to improve programs. Pine (1981) proposed a 
solution he called collaborative action research, whereby university 
faculty and school counselors cooperate in seeking solutions to practical 
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problems that arise in providing guidance and counseling services to 
students. In contrast to the usual situation in which researchers enter the 
school without the equal participation of counselors, research respon- 
sibilities are shared and counselors serve as co-authors of research 
reports. Such integration of research and practice draws on the university 
researcher's knowledge of methodological issues and procedures and the 
school counselor's familiarity with the issues to be investigated. Thus 
the strengths of both groups are used. 

School Counselors' Role in Research. School counselors also have 
an important role in research and evaluation. In this day of 
accountability, it is the counseling outcomes of their programs that are 
on the line. They need to set evaluation as a priority that is essential to 
the system support. They must reach out for assistance, if necessary, and 
welcome strategies that may strengthen their programs. 

Conclusion 

The bottom line is that we all need to work together. Standards need 
to be set either locally or by states so that counselors, administrators, and 
the general public understand what school counseling programs are 
today. Couaselor educators, in turn, need to teach to those standards. 
Counselors must be prepared to assume the role expected of them on the 
first day of practice. Counselor educators, state departments of educa- 
tion, and practicing school counselors must work in concert if we are to 
be credible as a profession. We are no longer trying to figure out who we 
are. We know. Now we must produce the research to convince the rest of 
the world that school counseling programs are vital in preparing our 
young people for their futures. 
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Chapter 5 

Student Development in 
Higher Education 

Cynthia S. Johnson 



Background 

The area of Student Services, or Student Development as it is 
currently called, is an important component of many counselor 
education programs across the country. From the first preparation 
program at Teachers College in 1913, the field has grown to over 100 
graduate preparation programs producing over 1200 graduates per 
academic year at the master's and doctoral level. 

The purpose of this article will be to provide a review of research as 
it relates to the important question of outcomes in Higher Education and 
in the profession. 

The Student Affairs Profession 

Student Development graduates perform multiple roles in 2- and 4-year 
colleges and universities. They counsel, advise, orient, and teach while 
working in Counseling Centers, Career Centers, Admission Offices, 
Residence Halls, Student Activities Offices, Orientation Programs, 
College Unions and any of over 30 out-of-class arenas on today's 
college campus. Their core curriculum includes counseling, management 
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and administration of higher education, and student development theory. 
Their roles include that of counselor, administrator and student 
development educator. 

The counselor role in this professional field varies. On the commu- 
nity college campus most counselors must have master's degrees, and 
their roles include that of adviser, career counselor, and personal coun- 
selor. On the 4-year campus most counseling centers are staffed by Ph.D. 
Counseling Psychologists from American Psychological Association 
(APA) approved programs. However, it is believed that all professionals 
in the field do some kind of counseling whether it is working with a 
student leader in academic difficulty, or with a residence hall "room- 
mate" conflict. 



Current Demand for Outcomes Research 

The topic of outcomes is one that is of major interest in Higher Educa- 
tion today as a result of: 

• the numerous reform reports critical of colleges and universities 

• legislative and taxpayer concerns about the "value" of higher 
education, and the billions of dollars it costs 

• new demands by accrediting bodies indicating a need for 
documenting student outcomes assessment 

With fiscal "downsizing" or "rightsizing" occurring currently in 
higher education in over 23 states, questions are being raised about the 
necessity of counseling centers, career centers, and highly paid admini- 
strators on college campuses. As a result of these factors at least 1 1 
states have mandated that assessment studies on the "value added" of a 
college education occur annually. 

The net result is that Academe is no longer able to hide behind the 
"Ivory Tower" and explain what it does in broad, vague terms such as 
"producing the educated person." This trend has forced student affairs to 
articulate more clearly what it is about, and to adopt new research strate- 
gies that document the results of specific interventions in terms that 
Boards of Trustees, parents and legislators can understand. 
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Current State of Research in Student Development 

George Keller, in an article in Change Magazine (1985), discussed the 
problems with research in Higher Education. He said that there is a 
preoccupation with research methods (referred to erroneously as 
methodology), a lack of research addressing the major problems of 
academe, and plentiful but scanty scholarship. He further stated that 
people responsible for decision making in higher education seldom read 
the research anyway. 

Since the 1960s, the amount of research in the field of Student 
Affairs has increased significantly, as have the professional journals that 
publish such research. Research in the field has been summarized under 
five major topical areas (Kuh et al., 1986): 

1. Behavior — achievement admission/matriculation, alcohol/drugs, 
addiction, persistence/retention, academic dishonesty, sexual 
activity and preference, and student activities including partici- 
pation in student government and other leadership functions 

2. Selected characteristics — aptitude, vocational, and educational 
aspirations and interests, attitudes/perceptions and expectations, 
commuters, educationally disadvantaged, international, fraternity/ 
sorority, graduate/professional, physically disabled, health/sui- 
cide, ethnic group (race, religion), socioeconomic status, transfer, 
veterans and women 

3. Student development — career/vocational, cognitive/intellectual, 
moral/ethical, social/emotional 

4. Instruction — developmental/remedial, compensatory/study skills 

5. Miscellaneous — finances, learning styles/preferences, living 
environments, campus ecology/student-faculty interaction, 
students' rights/legal issues/discipline, and an "other" category 

In a review of 1,189 articles published between 1969 and 1983 (Kuh 
et al., 1986) it was found that the research methods became more 
sophisticated over time with the majority of studies using multivariate 
techniques and being more theory-based. 

Current problems with research in this field include: 

• heavy reliance on psychological methodology to the exclusion of 
other fields of inquiry (anthropology-sociology, physics, etc.) 

• small, narrowly focused studies that ignore broader issues 

• too few studies that are replicable and generalizable 
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• lack of clearly measurable goals and priorities in the profession in 
general 

• loo few studies with respect to the diversity of today's college 
student populations 

• too little evidence that Student Development theory really works 

• insufficient agreement on what outcomes assessment means 

• too few longitudinal studies 

• insufficient funds from grants or external agencies to initiate 
meaningful large-scale studies of college students 

• too few scholar/researchers being nurtured in the field 

• and the corollary of the above— too many practitioners trying to 
juggle research agendas and the multiple daily demands of their 
work 

• a lack of role models and rewards for the utilization of the 
research that does exists in the field 

• evidence (Johnson & Steele, 1984) that research is not high on the 
agenda of some student affairs administrators 

While numerous problems exist with research in this field much 
progress has been made in the past 2 decades. Theoretical foundations 
for the field now provide a basis for measuring maturity and growth over 
time in psycho-social, cognitive, and moral and ethical aspects of a 
student's life. Instrumentation has been developed that has helped the 
field be more precise in addressing the question of outcomes. More 
sophisticated questions are being asked, and there are more professional 
forums and publications that allow for the debate and exchange of ideas 
and information. 



Major Research Findings in the Field 

This brief article will not attempt to summarize the major findings in the 
overall field. Instead it will address two major trends in student affairs 
research: (1) developmental theories and diverse populations and, 
(2) outcomes assessment 

Student Development Theory 

While Student Affairs was born at Harvard in the 1870s with the 
appointment of the first Dean, and developed a philosophy about the 
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maturation of the whole student in 1937, it wasn't until the late 1960s 
that the field of student affairs adopted a theoretical base. Drawing 
heavily from other disciplines, psycho-social theorists such as Erikson 
(1959, 1963, 1968), and Chickering (1969), cognitive theorists such as 
Kholberg (1969) and Perry (1970), and other developmental theorists 
laid the groundwork for this field as professionals assisted college 
students in their maturation and growth. 

While counselors, residence hall workers and other practitioners 
enthusiastically embraced this new theoretical orientation there were 
those that questioned the efficacy of the theoretical applications. 
Stamatakos (1987) called the adoption of these theories a "brass ring" 
embraced without systematic methodologies or thought given to the 
congruence of these theories with the professions philosophy. Bloland 
(1986) called student development a "new orthodoxy" adopted 
uncritically without waiting for research results suggesting that it 
worked. 

What is currently new in the research area in this field is an increase 
in the evidence that some student development theory applications are 
having a direct impact on college student outcomes (Pascarella & 
Terenzini, 1991). 

In a review of published empirical research between 1973-1987 
(Thrasher & Bloland, 1989) 27 research articles were analyzed. While 
the authors found the amount of empirical research on student 
development meager, evidence suggested that some programs and inter- 
ventions had a positive effect on student growth. For example, in their 
summary they stated that "typically students who participated in devel- 
opmentally designed courses advanced more on developmental scales 
then did other students enrolled in traditionally taught courses" (p. 553). 

In reporting the woric of Whiteley (1982) and Loxley and Whiteley 
(1986) students participating in the Sierra project, a residence hall 
program designed in part to increase levels of moral maturity, were 
found to "score significantly higher on the principled moral reasoning 
scale then did other students in a control group." 

In terms of psycho-social development some programs yielded sig- 
nificant increases in the areas of autonomy and development of purpose 
through use of developmental interventions. 

Pascarella and Terenzini (1991) produced further documentation 
about the research findings on the effectiveness of developmental theory 
in their book How College Affects Students. 
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While citing an impressive accumulation of evidence they also offer 
the following cautionary notes: 

• the evidence has a bias because it focuses largely on non-minority 
students of traditional college age (18-22), attending 4-year 
institutions full time and living on campus. 

• the samples vary from single-institution samples with only a few 
students to multi-institution, nationally representative samples 
with hundreds and often thousands of students 

In view of this they question the generalizability of some of the findings. 
In spite of these words of caution, theoretically-based empirical studies 
and developmental theory interventions clearly produce results. 

What is now needed is for these theories to be more thoroughly 
developed to provde additional information to professional preparation 
programs and to practitioners about the new majority on today's college 
campuses. Most of the theories that are currently being studied by 
graduate students were normed on a population that is now the minority. 

For example, some gender studies show that women's cognitive, 
moral and psycho-social development differs from that of men. Women 
now comprise 53% of today's college students. Students over 25 will 
soon be the majority if they are not already. Yet most of the existing 
theories were normed on traditional-aged white, male student popula- 
tions. 

In California students of color are already the majority in the 
community college system. Do current theories generalize to non-white 
populations? Should research be done to compare and contrast existing 
theories with different populations? Or should existing theories be "de- 
constructed" and new ones formulated? 

Clearly research on developmental theory outcomes will be a major 
agenda of the 1990s and beyond. 

Outcomes Assessment 

With an increasing segment of society asking what is the return on 
investment for a college education, counselors and counselor education 
programs need to develop more sophisticated methods of assessing out- 
comes. Problems, however, abound with the issue of research on assess- 
ment. There seems to be, for example, a lack of consensus on just what 
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"assessment" means (Terenzini, 1989). Is it a series of surveys, a self- 
study, or a review of the curriculum and co-curriculum? 

Further definitional issues exist when discussing assessment. What is 
its purpose? Is it for the enhancement of the learning environment or for 
accountability? What is to be assessed? Knowledge, skills outcomes, 
attitudes and values, or behavioral outcomes? Many student affairs pro- 
grams and institutions of higher learning have unclear missions and 
goals and unclear technologies that make assessment and discussion of 
desired outcomes difficult if not impossible. 

What is 'Value added"? Can value be added without positive change? 
And what do we mean by change? Ate change and impact the same? 
Pascarella and Terenzini (1991) note that: 

Just as the presence of change does not necessarily indicate the 
impact of college, so too the absence of change does not neces- 
sarily indicate the absence of college impact. One important 
consequence of college attendance may be to fix development at a 
certain level and prevent reversion or regression, (p. 565) 

Additionally, there are problems of organizational size and com- 
plexity, coordination, cost, methodology and measurement (Teranzini, 
1989). 

And finally Allen (1990) points out that most existing assessment 
instruments and methodology are based on a linear model of develop- 
ment and that development may occur through multiple pathways that 
are difficult to measure with existing methods. She suggests that we look 
into qualitative methods or Gleick's (1987) Chaos model which is being 
discussed and used in the field of science. 

In spite of the above-mentioned problems, many states have 
mandated assessment and have effectively developed comprehensive 
assessment programs measuring outcomes in terms of student growth 
and development in the out-of-class life of a college student. These 
annually conducted assessment programs have served to improve 
campus programs and satisfy external stakeholders on questions of 
accountability. 

In the State of Virginia student affairs administrators were active on 
the Governor's committee that mandated assessment and were able to 
influence its design. Most notable among the model programs on 
assessment are the ones at George Mason University and Longwood 
College in Virginia. On these campuses assessment has been integrated 
with the goals, priorities and budgeting for Student Affairs departments, 
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and data being collected over time will assist decision makers and 
benefit students. 

The assessment issue has forced Student Affairs and Academic 
Affairs to define more precisely what they are about and to provide 
evidence of accomplishment. It has shaped many of the campus research 
agendas and resulted in a number of national- and state-based instru- 
ments for the puipose of measurement 

Research Design Trends 

Clearly the major trend in research design is the emphasis on qualitative 
research in this professional field. As stated in a recent Journal of 
College Student Development (Caple, 1991) "For too long doing 
research in student affairs, like many other fields, has largely been 
restricted to the use of one general design structure — quantitative 
methods. The qualitative method of thinking about and conducting 
research is an important and viable alternative" (p. 387). 

What is necessary is for counselor education faculty to become more 
informed about the methodology of qualitative research, and for the 
profession itself to adopt more accepting attitudes toward naturalistic 
and qualitative methods of inquiry. 

Fortunately, an increasing number of professional forums and publi- 
cations are featuring programs and models for qualitative research that 
will help faculty and professionals alike become more adept with this 
research method. 

While not all questions lend themselves to qualitatively designed 
research, Patton (1990) and Goetz and LeCompte (1984) consider 
qualitative methods to be superior to other research methods for certain 
situations, such as attempting to understand complex processes like 
change in college students. Many professionals in student affairs have 
concluded that qualitative research methods are especially appropriate 
for this field because they allow one to take an in-depth look at questions 
from the perspective of the student 

Attinasi (1992) believes that we cannot make progress in under- 
standing college student outcomes unless we adequately understand 
what meaning college has for students themselves. His own research in 
this area, using in-depth, phenomenological interviewing as a qualitative 
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research technique to look at Latino students, provides one such model 
for doing so. 

Conclusion 

With downsizing and diverse populations as the two major issues facing 
higher education today, the agenda is clear. More evidence of the 
outcomes of the counseling, advising and educational process must be 
provided using research methods that can assist the student in growing 
and becoming successful, and can provide more then the traditional 
"bean counting" data previously provided by some professionals. 

Too little is known about many of the populations that are rapidly 
becoming the "New Majority" on our campuses. Much more research is 
needed on the ways Latinos, African Americans, Asian Americans and 
Native Americans learn, grow, and make decisions. For example, is the 
independent, linear decision making model used in career development 
appropriate for these populations? Are we measuring new populations 
against Euro-centric models of traditional-aged white males and finding 
them different or less equal? What do we know about the counseling 
needs of the Lesbian, Gay, Bi-sexual student, and are we translating that 
knowledge to the graduate classroom? In areas where much progress has 
been made on the research agenda, for example adult development, are 
we integrating that knowledge in practice, or are we still teaching about 
more traditional populations? 

Another future agenda would include developing a more global 
perspective for research. More studies that compare and contrast 
counseling, career counseling, decision making and issues of self-esteem 
across continents would inform practice and help prepare students for 
the international community in which they will work. 

Finally, faculty must encourage graduate stuJents to move beyond 
the narrow scope of their own research and publication agenda to ask 
generalizable "so what" questions that can benefit the profession and can 
be replicated. 

All professionals need to be encouraged to use qualitative research 
methods. Only by looking at the many "big picture" issues facing 
today's new college student can we practice, become informed and 
enhance the important outcomes documented. 
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Additional Recommended Resources 

Barr, M., & Upcraft, L. (1990). The future of student affairs. San 
Francisco: Jossey Bass. 

An excellent overview of the many areas where research is needed by 
the profession. 



Jacobim, A., & Ayala, E (1987). College student outocmes assessment. 
Washington, DC: ASHE-ERIC. 

The book discusses a variety of ways to approach this topic and 
documents why some of the approaches fail. 

American College Personnel Association. (1991, September). Journal of 
College Student Development (Special Edition). 

This edition is devoted to the topic of qualitative research and 
provides model studies for the practitioner and researcher alike. 

NASPA Publications. (1992). Puzzles and pieces in wonderland: The 
promise and practice of student affairs research. Washington, DC: 
Author. 

The most current review of the research literature to date. 

PascareUa, E., & Terenzini, P. (1991). How college effects students. San 
Francisco: Jossey-Bass Publishers. 

A monumental review of the research on outcomes over the past 
twenty years. 
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Chapter 6 

Marriage and Family 
Counseling 

Jon Carlson 
Introduction 

More and more counselors and therapists are working from a 
systems perspective. Family and couples counselingAherapy is a 
treatment approach which views and treats the family as a system. The 
system has properties all its own: a set of rules, assigned and ascribed 
roles for its members, an organized power structure, intricate overt and 
covert forms of communication, and elaborate ways of negotiating and 
problem solving that permit various tasks to be performed effectively 
(Goldenberg & Goldenbeig, 1991). Family counselors/therapists engage 
in the clinical practice of treating individuals, couples, or families. 
Regardless of the number of clients being treated, the family counselor 
conceptualizes problems in terms of the systems perspective. The family 
counselor therefore focuses on the context in which individuals exist and 
intervene on the relational level. Research support for family and cou- 
ples woik is indicated for a wide variety of situations. Proponents come 
from the ranks of medicine, social work, psychology, counseling, and the 
clergy. 
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The efficacy of marital and family therapy has been well supported 
by clinical research. Over the past 25 years there has been a steady 
increase in the amount of systematic, empirical study of marital and 
family therapy (Gurman, Kniskem, & Pinsof, 1986). A significant divi- 
sion, however, exists between researchers and clinicians. According to 
Gurman and Kniskem (1992): 

The clinician-researcher gap is not only in terms of the usual 
differences between these groups (e.g., nomothetic vs. idiographic 
richness and observational rigor vs. inferential richness^ but also 
in terms of the most basic of epistemological premises. Thus, 
although some argue that traditional quantitative research 
methods are generally adequate to the task of studying complex 
systemic treatment phenomena, others (e.g., Tomm, 1983) argue 
that because causality cannot be established and "objectivity" is 
an illusion, the hypotheticodeductive methods of the physical 
sciences are inappropriate for the study of family therapy, (p. 67) 

Research support and understanding of supervision and training in 
family counseling and therapy have also presented many challenges. The 
purpose of this paper is to review the major research results, trends, and 
developments, future research, and implications for counselors and 
counselor educators. 



Analysis of Research 

Researchers in couple and family therapy/counseling attempt to under- 
stand this complex therapeutic process. For the past 40 years research in 
psychotherapy has attempted to investigate the therapeutic process in 
order to develop more effective methods of treatment Previous research 
tended to focus on the legitimacy of psychotherapy. According to 
VandenBos (1986) "by 1980 a consensus of sorts was reached that 
psychotherapy, as a generic treatment process, was demonstrably more 
effective than no treatment" (p. 111). According to Smith, Glass and 
Miller (1980): 

1. Overall, psychotherapy is effective, with the average patient 
treated (functioning) better than 80% of those who need therapy, 
but remain untreated. 

2. Well -controlled comparisons of different approaches to therapy 
yielded no reliable differences in effectiveness of behavioral and 
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verbal therapies. Thus, there was little evidence of differential 
effectiveness among the different therapies. 

Researchers have moved beyond the single-focus outcome studies 
and have turned their attention to comparative outcome studies, concen- 
trating on the relative advantages and disadvantages of alternative 
treatment strategies for clients with different sets of problems. Explora- 
tions of process variables are taking place so that differential outcomes 
from various therapeutic techniques can be tentatively linked to the 
presence or absence of specific therapeutic processes. This shift is due in 
part to the economically-driven therapy marketplace with PPOs, HMOs, 
and DRGs demanding focused, brief, scientifically justified interven- 
tions. The day may not be far off when third party payers will reimburse 
only those treatments demonstrating efficacy. 

The research-practice connection is weak in family counseling and 
therapy and must be strengthened (Liddle, 1991). Research touches 
every counselor supervisor, counselor educator, and trainee. Liddle 
believes that family counselors need to (1) decide how to use the basic 
research on the psychological, social, and physiological processes, and 
(2) use family counseling research in clinical practice. Research, 
however, has failed to play a very prominent role in the development of 
family counseling and therapy (Bednar, Burlingame, & Masters, 1988; 
Werry, 1989; Goldberg & David, 1991). Counselors do not seem to use 
research in their daily work and defend this behavior by claiming that: 

(a) questions addressed in research are irrelevant to clinical practice; 

(b) treatments, measures, populations, and variables investigated insuffi- 
ciently represent actual clinical realities; (c) data analyses overempha- 
size group means and tell little about clinical significance or individual 
cases; and (d) journal articles on treatment research are written in a way 
and appear in forms that are inaccessible (Cohen, Sargent, & Sechrest, 
1986; Hayes & Nelson, 1981; Morrow-Bradley & Elliot, 1986). Liddle 
(1991) feels that the attitude toward and values about research must be 
changed and isolates 10 key dimensions that will accomplish this 
change. 

Major Research Findings 

Gurman and Kniskem (1978) reviewed family therapy outcome and 
process research and drew the following conclusions: 
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1. Family treatment is more effective than no treatment. This is 
manifest in studies that contrast family and marital treatment to 
no-treatment control groups. The rate of improvement from 
marital and family treatment is about 66%, approximately the rate 
found in most therapy outcome studies of individual therapies. 

2. While many patients benefit from family therapy, some patients 
(estimated at 5%-10%) worsen or suffer negative effects from 
family intervention. The deterioration rate is lower than that 
estimated for individual therapies. 

3. There are a few areas in which some evidence suggests that 
family/marital therapy may be the preferred treatment interven- 
tion strategy. To say that in these instances family therapy is the 
treatment of choice may be putting the case too strongly, but com- 
parative outcome studies favor family/marital treatment in the 
following situations: (a) marital treatment for marital prob- 
lems/conflict; (b) conjoint sex therapy for sexual dysfunctions 
that involve the interaction context; (c) family therapy for the 
treatment of childhood acting out such as aggressiveness and 
adolescent juvenile delinquency. Other situations in which evi- 
dence indicates that the family/marital format is effective, but for 
which further comparative research is needed, include family 
treatment of asthma and anorexia, adult drug addiction, marital 
treatment when one spouse is depressed, and adjunctive treatment 
when an adolescent or young adult is schizophrenic. 

In her review of the studies of effectiveness of marital and 
family therapy, Thomas (1992) cited research supporting marital/ 
family treatment for: 

• improvement of healthy functioning 

• marital distress 

• divorce 

• childhood antisocial behaviors and conduct disorders 

• childhood emotional and behavioral problems 

• childhood tics 

• anorexia and bulimia 

• psychosomatic illnesses of children and adolescents 

• school problems 

• juvenile delinquency 

• drug addiction 

• alcoholism of spouse 
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• depression 

• schizophrenia 

• multiple personality disorder 

• closed head injury 

• sexual problems 

• family conflict 

4. While the various schools of family intervention (e.g., structural 
strategic, behavioral, Adlerian) often claim or imply superiority' 
there is very little controlled research on the approaches while 
holding the format of treatment constant. This research would 
probably yield little differential information even if conducted 
Rather, the field is going to manualization of therapies with 
precise specification of techniques, and when these are applied 
often m usefully integrated mixtures, positive results are forth- 
coming. 

As for the question of what treatment for what problem, Gurman and 
Kwskern (1981a. 1981b) and Gurman, Kniskern, and Pinsof (1986) 
ottered some conclusions based on their literature search: 

1. Conjoint treatment for marital discord is clearly the method of 
choice over the individual, collaborative (each spouse sees a 
different therapist), or concurrent (one therapist treats marital 
partners in separate sessions) approaches. 

2. The beneficial effects of nonbehavioral and behavioral marital/ 
family therapy often occur in treatment of less than 20 sessions 

3. Compared with no treatment, nonbehavioral marital/family 
therapies are effective in approximately two-thirds of all cases 

4. Behavioral marital therapy (BMT) is about as effective for mini- 
mally distressed couples as nonbehavioral methods, somewhat 
less so when severe dysfunction is involved. 

5. Increasing a couple's communication skill, however achieved is 
the essence of effective marital therapy. 

6. Conjoint behaviorally-oriented sex therapy should be considered 
the treatment of choice for sexual problems, especially when 
severe nonsexual problems do not exist. 

In their surveys, these reviewers found family therapy to be more 
effective than individual therapy, even for problems that seem more 
intrapsychic in nature than interpersonal. More specifically 
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7. 
8. 



Structural family therapy appears to be particularly helpful for 
certain childhood and adolescent psychosomaUc symptoms 
As for the relative efficacy of behavioral and nonbehavioral 
approaches, no conclusions are justifiable on *e ^basisof 
published research. However, either strategy is clearly preferable 
to no treatment at all. , . f „„ lWoiw 

9 No empirical evidence yet exists for the supenonty oleotherapy 
' over single therapist interventions with couples or families. 

10 S Xagement training, a behavioral technique, produces 
more favorable results with children engaged in antisocial 
behavior than do nonbehavioral techniques. 

The data collected by Gurman and Kniskem (1981a. 1981b) also 
support the following: 

11 Short-term and time-limited therapies appear to be at least as 
effective™ treatment of longer duration Moreover most of die 
Positive results of open-ended therapy were achieved in less than 

12 slvemlmrital and family ''enrichment'; programs appear to 
have promise as useful strategies in famUy hvmg (Gi bhn 1986)^ 

13 The more members of the family involved in the family therapy 
me better the outcome seems to be. This is especially true when 
the child is the identified patient 

14 mrapfs relationship skills have major impact on the outcome 
of marital and family treatment regardless of the "school or 
orientation of the therapist. Training programs must focus on 
Zh conceptual-technical skills and relationship skills for 
beginning therapists. 



Trends and Development in Research 

Numerous research designs exist that will j^ij^^^ 
effectiveness of a particular type of counseling. The APA Commission 
^SS^ (1982) placed research designs on « t conunuum from 
?owei Sri Wes e.g., one-shot case studies) through tugher levels to 
evel Vtype^g.. multivariate designs) which are increasingly 
implex deduce confounding of variables, and increase the level of 
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knowledge obtained from the research effort. The following are criteria 
that apply to any psychotherapy outcome study: 

1. Assignment to treatment condition by random assignment, 
matching, etc. 

2. Measurement of the status of the client/family on foci of the 
intervention before and after treatment. 

3. Matching of treatment and control groups on major independent 
variables such as therapists' experience and competence level, 
enthusiasm and belief in treatment they are administering, equal 
amounts of time in therapy, etc. 

4. Assessment of clients not only at the end of treatment, but also at 
follow-up period (often 6 months, and/or 12 months) in order to 
assess the durability of treatment effects. 

5. Description of the therapeutic techniques and strategies in suffi- 
cient detail (i.e., often some form of treatment manual) in order to 
teach therapists the techniques and to see if the treatment as 
described is being delivered. 

6. Assessment instruments carefully selected in order to: assess 
individual and family change; assess those areas targeted by the 
two treatments as foci for change either as mediating or final 
results; judge change from the multiple perspectives of individual, 
couple, family, therapist, independent observer, assessment of 
both positive and negative outcomes. 

7. Statistical analysis that would assess group differences with mean 
scores, percentage of clients/families clinically improved in both 
groups, etc. 

8. Data on concurrent treatment (Glick, Claikin, & Kessler, 1987) 



Future Research 

Many researchers are calling for an end to "horse race" research 
(Jacobson, 1991) pitting different theoretical approaches against each 
other with the hope of discovering which one is best. The futility of this 
type of approach has been addressed by Maher (1988). There is 44 no 
established theory of psychotherapy that declared bankruptcy because of 
research that failed to confirm, disconfirmed, or falsified its theoretical 
propositions" (p. 694). Moreover, "it has probably never happened that 
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the proponents of a theory of psychotherapy have spelled out the 
research grounds under which they would probably forswear their theory 
or renounce important theoretical propositions" (p. 695). However, 
Gurman (1991) made an interesting point when he indicated that 
methods of therapy need to have a reasonable degree of evidence of their 
efficacy in order to warrant further study. Once this has occurred, further 
intramodel studies can occupy the largest share of research efforts. This 
view has been challenged by Jacobson (1991) who has called for a push 
for discovery-oriented (vs. confirmatory) therapy research. (This is 
interesting in that Jacobson has been the pioneer in intramodal research.) 

Moon, Dillon, and Sprenkle (1990) have advocated a qualitative 
research paradigm for family therapy, as they feel that this is congruent 
with systems theory. This model may also enrich process, discovery, 
feminist, and outcome research. Qualitative methods provide contextual 
data that can enrich the interpretation of quantitative outcome studies. 
For example, Jacobson, Schmaling, and Holtzworth-Munroe's (1987) 2- 
year follow-up of a component analysis of behavioral marital therapy 
(BMT) containrd both a traditional, quantitative, before-after 
experimental study (Kerlinger, 1986) and a qualitative, structured inter- 
view study. The quantitative portion of the study revealed the differences 
between treatments which had emerged at the 6-month follow-up, 
disappeared by the 1-year follow-up, and did not reappear at the 2-year 
follow-up. 

From the qualitative portion of the study the researchers were able to 
generate hypotheses as to why the treatment gains slipped away. These 
hypotheses, in turn, were used to develop an expanded version of BMT 
which included booster sessions to extend the effects of the initial treat- 
ment. Thus, hypotheses generated by the qualitative portion of an 
outcome research project helped BMT researchers interpret quantitative 
results and create a treatment modification that can be tested by future 
researchers. The work of Jacobson et al. (1987) clearly illustrated the 
complementarity of the quantitative and qualitative paradigms and the 
need for both types o f research in some outcome studies. 

Feminists dispute the appropriateness of quantitative methods of 
doing social science research. Thompson (1992) reviewed the feminist 
literature on both methods and contended that future research needs to 
be adapted and elaborated to serve feminist family studies. She also 
discussed two prevailing ethical concerns: Whose interests are served by 
research, and how the subjectivity and authority of research participants 
can be preserved. 
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Finally, Williams (1991) proposes a blueprint to close the gap 
between research and clinical practice. Williams believes that research 
should be more relevant and directed toward human needs. A model 
taken from private Industry and based on a marketing and developmental 
perspective is proposed: 

Stage L Define consumers; develop an understanding of the 

consumer needs 
Stage 2. Generate new ideas 

Stage 3. Develop a workable concept that targets a consumer need 
Stage 4. Develop a prototype with a comparative advantage under 

limited conditions 
Stage 5. Develop a product with a comparative advantage under 

real conditions (p. 356? 

As with each of the previous suggestions, it seems important to 
consider switching the focus of research to decrease the gap between 
research and clinical practice. 

Implications for Counseling and Counselor Education 

Kniskern and Guiman (1979) made the following startling assertion: 

We are unaware of empirical study of either process or outcome 
of training programs in the family field. There now exists no 
research evidence that training experiences in marital-family 
therapy in fact increase the effectiveness of clinicians, (p. 83) 

These authors went on to decry the lack of data about other aspects of 
the training process. There is, for example, no evidence concerning: 
(a) the importance of prior training in family therapy for persons enter- 
ing training programs, (b) criteria for selecting trainees, (c) preferability 
of training in one theoretical orientation versus many, or (d) the value of 
experiential methods such as personal therapy, role-play, and family-of- 
origin woric for the trainee (Avis & Sprenkle, 1990, p. 241). 

Nearly a decade later Kniskern and Gurman (1988) offered an update 
on their research recommendations. Their characterization of this area of 
the field became even more critical in the citation of a "remarkable 
empirical ignorance of what types of activity or styles of intervention 
should be fostered in trainees" (p. 372). Kniskern and Gurman 
discussed the irony in the fact that, although many of the field's pioneers 
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worked in settings in which training was primary, the empirical evalua- 
tion of family-therapy training has only recently been appreciated as 
important enough to undertake. These authors recommended a five-step 
process of training program evaluation: 

1. Identify and specify training goals 

2. Develop the training model 

3. Develop measures that evaluate training-induced change 

4. Demonstrate the expected change in trainees 

5. Demonstrate that trainees who have shown the expected changes 
on the measures are better able to help families than are those 
who did not have the training 

Why has research on family therapy training developed so slowly 
when outcome research of family and couples work is mushrooming? 
The reasons for this delay probably lie in the difficulty inherent in this 
type of research. In one of the first empirical evaluations of a family 
therapy training program, Hicker and Pinsof (1984) identified the diffi- 
culties in evaluating family therapy training: (a) the complexity of the 
type of changes being measured; (b) the lack of a standard stimulus 
(families vary) against which to measure trainees* skills; (c) the lack of 
adequate and appropriate instruments for measuring change; and (d) the 
lack of reliable knowledge about which therapist skill or behavior is 
associated with positive family therapy outcomes. 

Avis and Sprenkle (1990) found only 15 studies in their follow-up to 
Kniskern & Gurman (1979). From these studies the authors made the 
following conclusions: 

1. We now have several instruments with some degree of validity 
and reliability which can distinguish between beginning and 
advanced therapists, to measure the acquisition of conceptual 
and/or intervention skills, and to offer feedback to therapists on 
their in-therapy behavior. 

2. There is evidence that various forms of family therapy training 
can produce an increase in trainees' cognitive and intervention 
skills, although the latter is less certain because intervention skills 
have never been measured in actual therapy sessions. 

3. Cognitive and intervention skills may develop independently of 
one another. 

4. In-service family therapy training programs for agency and staff 
may be an effective way to increase agency service to families. 

5. Beginning assessment skills may be as effectively taught using 
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traditional classroom methods as by using more expensive experi- 
ential methods. 

6. Sequencing of training activities may be a significant variable in 
the acquisition of family therapy skills, (p. 260) 

These authors questioned the adequacy of the research in terms of: 
sample size and random sample; control for trainee variables; use of 
control or comparison group; standardized training; follow-up; trainer 
variables; testing procedures used; and trainer-investigator nonequiv- 
alence. 

Avis and Sprenkle (1990) recommend the following guidelines for 
future research in family counseling and therapy training: 

1 . There is a need for controlled research on both training and super- 
vision. 

2. More research should focus on replication of the better existing 
studies in order to confirm (or disconfirm) trends and tentative 
findings. 

3. Develop more valid and reliable instruments with which to 
measure trainees' skills. 

4. Pay more attention to evaluating training in terms of its impact on 
therapeutic outcome. 

5. Various design improvements are necessary before the research 
on training may be considered methodologically adequate. 

6. Comparative studies will be increasingly important for answering 
the specificity question — i.e., the question of what training is 
effective when, for whom, under what conditions, and for what 
kind of clinical situation. 

Overall, evaluating the outcome of family therapy training is a 
fledgling research endeavor of tremendous importance to the field. As 
family therapy grows it is imperative to have training procedures based 
on a sound empirical foundation. 

Implications for counseling and counselor education are: 

1. Counselor educators need to train and instruct while utilizing 
methods and strategies that have empirical support rather than the 
traditional "seat-of-thc-pants" style of training. 

2. Counseling and counselor education need to become active in 
conducting outcome research. Although we can extrapolate from 
other related research bases, counseling is different and needs its 
own research. We need to know what is effective and why. 
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3. Consideration must be given to increasing "consumer driven' 
research. 



Conclusion 

Gurman (1991) indicated that the problems of research and research 
design may be only minor. 'The major consideration, in my view, is that 
therapists simply do not choose (or switch to alternative) modes and 
methods of therapy (whether marital, individual, or whatever) on the 
basis of even rather convincing bodies of empirical evidence. We choose 
therapies because we identify with important teachers; we find some 
ways of thinking that are intellectually and/or aesthetically appealing (in 
ways we cannot always articulate); we find ourselves more comfortable 
with some ways of working and downright anxious or awkward with 
others, etc., etc. Even HMOs, PPOs, and DRGs will not amass enough 
power to overwhelm such personal proclivities" (p. 404). 
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Liddle, H. A., Breunlin, D. C, & Schwartz, R. C, (Eds.). (1988). 
Handbook of family therapy training and supervision. New York: 
Guilford Press. 

A thorough presentation of the field of family therapy training 
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The two most useful references are the Journal of Family Psychology 
and the Journal of Marital and Family Therapy. Both publish research 
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Chapter 7 



A Review of the Counseling 
Outcome Research 



The efficacy of counseling practices, procedures, and approaches is of 
particular interest to counseling professionals. In fact, throughout the 
history of the counseling profession there has been ongoing inquiry and 
debate regardifig the efficacy of counseling (Highlen & Hill, 1984; 
Whiston & Sexton, 1993). Answering the questions regarding the most 
effective elements of counseling is a complex and perplexing task. 
Practicing counselors make numerous decisions regarding practices and 
procedures, each of which has an impact on the outcome of their work. 
The counselor educators who may train these practitioners are faced with 
similar decisions regarding which techniques to include in program 
curricula. However, what is the basis upon which these practice and 
training decisions are made? 

The counseling outcome literature, one source of information 
regarding counseling efficacy, has had little impact on counseling 
practice and training. Cohen, Sargent, and Sechrest (1986) found the 
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instrumental use of counseling research by psychologists to be an 
infrequent occurrence. In a national survey of counselor educators, 
Sexton and Whiston (1992a) found that while educators rated their 
knowledge of the outcome research as high, their understanding of this 
research was often inaccurate. Woolsey (1986) even suggested that even 
many counseling students are antagonistic towards research. Since 
counseling outcome research is designed to evaluate counseling and 
assess the factors that have ti\e greatest impact on positive outcome, it is 
a potentially important source to which counseling practitioners, 
counselor educators, and counselors in training may turn in an attempt to 
understand the effectiveness of different interventions and theoretical 
perspectives. In addition, this research could be an important source 
from which to make the myriad of practice and training decisions facing 
counseling professionals. In fact, Lambert (1991) argued that without a 
thorough knowledge of outcome research a practitioner may not be able 
to serve in the client's best interest 

One explanation for the minimal impact of the empirical research on 
practice is that counseling effectiveness studies have been plagued with 
numerous methodological problems that often have made integration of 
this work into counseling practice difficult (Luborsky, 1987; Persons, 
1991; Sexton & Whiston, 1991). The restraints of control required by 
traditional quantitative research designs have resulted in many studies 
conducted in laboratory or analog situations that tend to reduce the 
natural complexity of the counseling process and limit generalizability 
of the results (Kazdin, 1986). Similarly, the choice and adequacy of the 
measures of client change have been questioned in terms of what 
constitutes therapeutic change, how much change is meaningful, and 
from whose perspective change should be assessed (Highlen & Hill, 
1984; Kazdin, 1986; Lambert & Bergin, 1983). In addition, for some 
practitioners the demands of daily work with clients leaves little time for 
consuming the multitude of research studies published in the wide 
spectrum of journals and books. 

Recently, three important changes have made the counseling 
effectiveness research a fruitful and important area to which counseling 
practitioners and counselor educators may turn (Luborsky, 1987). First, a 
number of recent literature reviews have taken advantage of the 40-plus 
years of clinical research (Lambert, Shapiro, & Bergin, 1986; Luborsky, 
Crits-Christoph, Mintz, & Auerbach 1988; Orlinsky & Howard, 1986). 
Thus, results can now be assessed across various studies in multiple 
settings. Second, the creation and sophistication of meta-analytic 
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research approaches have provided a way to combine very different 
types of studies and methodologies so that trends across laige numbers 
of investigations can be identified and specific effect sizes calculated 
(Kazdin, 1986; Smith, Glass, & Miller, 1980). In addition, box score 
tabulations that are the basis of qualitative research reviews result in a 
systematic summary of studies. Finally, there have been a number of 
recent, large-scale empirical studies that investigated the counseling 
process in clinically realistic settings and with methodologically sound 
procedures (Elkin et al., 1989; Luborsky et aL, 1988; Quality Assurance 
Project, 1983). 

The purpose of this chapter is to provide an integrated summary of 
major research trends and common elements of the counseling process 
that contribute to successful client change. The primary goal is to 
synthesize those findings and clinical studies into a form that is 
applicable to both counseling practice and training, and discuss the 
implications of this work for both areas. The discussion will first review 
the counseling outcome literature concerning the most basic question: 
Does counseling woik? If counseling works then what elements of the 
counseling process contribute to its success? In this regard, five 
components of the counseling process will be examined including: 
existing factors, the counseling relationship, session factors, specific 
models and techniques, and client experiences that occur within 
counseling. 

Figure 1 is a graphical representation of proposed relationships 
among these components and client change that will be used to organize 
this review (Sexton & Whiston, 1991). The bidirectional arrows indicate 
that like Highlen and Hill (1984) and Strong and Claibom (1982), these 
components of counseling are considered to be interactional and 
reciprocal. More comprehensive explanations of these elements are 
included in subsequent sections of this chapter. 

Counseling Outcome 

Initially the most important question is whether counseling is effective. 
Early studies of clinical effectiveness were based on non-systematic case 
study evidence and led to conclusions that lacked both reliability and 
validity (Heppner, Kivlighan, & Wampold, 1992). These difficulties 
with the counseling research process no doubt fueled the concerns of 
many, including Eysenck (1952), who repeatedly claimed that there was 
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little evidence for the effectiveness of counseling. He contended that 
counseling was not only ineffective but might even have a detrimental 
effect on clients who would have improved on their own. 

The evidence accumulated over the last 40 years is relatively clear: 
Counseling is a process from which most clients who remain involved 
for at least a few sessions benefit When counseling effectiveness is 
calculated by determining the number of clients who improved, the 
results are amazingly similar across various studies. On the basis of both 
client and counselor ratings, approximately 22% of clients made 
significant gains, 43% made moderate changes, while 27% made some 
improvement (Luborsky et al., 1988; Mintz, 1977; Rosenthal, 1983). In a 
major review of this literature, Lambert et al. (1986) suggested that on 
the average 65% of clients could be expected to make improvements 
while in counseling. 
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Counseling efficacy is also apparent when the amount of client 
improvement is calculated. The first meta-analytic studies (Smith & 
Glass, 1977; Smith et al. v 1980) found an average effect size of .85 
standard deviation units over 475 studies. A number of subsequent 
researchers reanalyzed these data (Andrews & Harvey, 1981; Landman 
& Dawes, 1982) and reported average effect sizes from .72 to .90 
respectively. Other researchers (Quality Assurance Project, 1983; 
Shapiro & Shapiro, 1982; Steinbrueck, Maxwell, & Howard, 1983) 
analyzed the effects of different samples of studies reporting average 
effect sizes ranging from .42 (Prioleau, Murdock, & Brody, 1983) to 
1.22 (Dush, Hirt, & Schroeder, 1983). 

In a recent clinical study, Luborsky et al. (1988) found psycho- 
analytic treatment to have similar effect sizes ranging from .69 with a 
measure of health symptoms to .80 based on a symptom checklist. 
Lambert et al. (1986) concluded that the average effect associated with 
psychological treatment is close to one standard deviation unit. On the 
basis of the standards developed by Cohen (1977) for understanding 
empirical findings in the behavioral sciences, the effect sizes due to 
counseling can be considered to be quite large. 

While these results are compelling, two additional factors are also 
important to consider when judging whether counseling is effective. 
Effective counseling should help clients make improvements above and 
beyond those that would naturally have occurred without intervention 
(Garfield & Bei^in, 1986). Berlin (1971), Lambert (1976), and Bergirf 
and Lambert (1978) suggested that while some clients do improve on 
their own, in comparison, those in counseling improved more and at a 
faster rate. The reported gains produced by counseling also are greater 
than "placebo" improvement rates (Andrews & Harvey, 1981; Landman 
& Dawes, 1982; Quality Assurance Program, 1983; Smith et al., 1980). 
In addition, effective counseling should result in client change that is 
maintained over time. According to the outcome research, changes made 
by clients continued to be maintained at follow-up (Andrews & Harvey, 
1981; Landman & Dawes, 1982; Nicholson & Berman, 1983; Smith et 
al., 1980). 

Unfortunately, it is also relatively clear that some clients do get 
worse during their work with a counselor. The reported negative effects 
ranged from 6%-7% (Orlinsky & Howard, 1980) to 11.3% (Shapiro & 
Shapiro, 1982). Yalom and Lieberman (1971) suggested that these 
negative effects may not have been due to the counseling process itself, 
but instead were due to actions of certain counselors. According to 
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Lambert et al. (1986), counselors who let the client ramble and did not 
focus the counseling session, failed to identify important therapeutic 
issues, and/or failed to integrate these issues and themes into counseling 
contributed to negative client outcomes. 

Summary: Counseling Outcome 

These counseling outcome research findings are so compelling that the 
debate regarding whether counseling is effective is no longer a major 
issue (Lambert, 1991). Clients who remain involved for at least a few 
sessions experience significant improvements that are greater than those 
which could be attributed to either a placebo condition or improvement 
due to time. In addition, these changes appear to be maintained over 
time. While some clients get worse, it appears that this lack of 
improvement may not be the result of the counseling process per se, but 
instead may be due to actions of the counselor. Lambert et al. (1986) 
suggested that if the skillfulness of the counselor had been controlled in 
these outcome studies it is possible that the effects of counseling on 
client change could have been even greater. Thus, the more important 
questions now involve identifying which aspects, methods, and factors 
make significant contributions to the changes clients make in counseling. 

Factors that Influence Counseling Outcomes 

Given that counseling has the potential to help clients make 
improvements, it is important to consider the elements that may 
constitute the counseling process. Factors that influence counseling 
actually begin even before the first interaction between client and 
counselor. Both client and counselor enter counseling with certain 
characteristics that may affect the process and outcome of the 
counseling. Yet to be answered are the questions concerning the degree 
of influence that these existing factors have on the outcome of 
counseling. 

Client Characteristics 

In a review of over 25 years of clinical research, Frank (1974) claimed 
that among the most important determinants of long-term counseling 
improvement are client-related factors or characteristics. A variety of 
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demographic characteristics have been the focus of empirical inquiries 
concerning these client factors. For example, both educational level and 
intelligence are positively associated with successful counseling 
outcome (Garfield, 1986; Luborsky et al. r 1988). On the other hand, 
even though women arc more likely to seek counseling (Cbllier, 1982), a 
client's gender is usually unrelated to outcome (Jones & Zoppel, 1982; 
Luborsky, Mintz, & Christoph, 1979). Interestingly, Carpenter and 
Range (1983) found that male clients are less likely than female clients 
to terminate once they have begun counseling. Finally, Luborsky et al.'s 
(1988) review also indicated that the majority of the studies did not find 
a relationship between age and counseling outcome, which is consistent 
with the results of Smith et al.'s (1980) meta-analysis. 

Determining the relationship between a client's race and counseling 
is a somewhat more complex and controversial. On the one hand, Jones 
(1982) found counseling outcome to be unrelated to the race of the 
client. However, Garfield (1986) noted that race was correlated with 
socioeconomic level, and Casas (1984) contended that one cannot 
examine race without incorporating sociocultural characteristics such as 
socioeconomic status. There are some studies that indicated that 
minority clients had a tendency to terminate early from counseling more 
frequently than whites (Acosta, 1980; Sue, 1977; Vail, 1978). On the 
basis of more recent research, Abramowitz and Murray (1983) 
maintained that the empirical support for premature minority teimination 
may not be as strong as previously believed. 

While the issue of race alone may not be significant in counseling 
outcome, understanding the role of ethnicity is more complex. For 
example, studies on minority utilization of counseling services have 
further complicated the picture. Abramowitz and Murray (1983), 
Billings and Moos (1984), and Sattler (1977) suggested that minorities 
underutilized counseling, while Atkinson (1985) argued that the results 
of studies on use vary according to the sample and methodology 
employed. Atkinson (1985) further stressed that patterns seem to vary 
within ethnic groups, and general statements about minority use may be 
unwarranted. Neimeyer and Gonzales (1983) found that white clients 
were more satisfied with treatment than non-white clients. Reviewing 
archival studies, Atkinson (1985) concluded that when compared to 
white clients, ethnic minorities received not only different but less 
preferred forms of treatment (such as not being referred for individual 
counseling or being seen by a nonprofessional). 
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Besides demographic characteristics, there are additional psycho- 
logical aspects of clients that are related to counseling outcome. 
Luborsky et al. (1988) found that the client's general level of emotional 
health was the best predictor of positive outcome. They also noted that 
this finding was consistent with other research that indicated that the 
severity of diagnosis was inversely related to positive outcome. 
However, Garfield (1986) claimed that adjustment and personality may 
not actually be predictive of outcome due to methodological problems 
because studies differed in how they defined and measured the 
constructs of adjustment and personality. 

Some studies focused on the initial emotional state of the client. In 
general, the greater the level of initial client anxiety, the better the 
outcome (Kemberg et al., 1972; Lemer & Fiske, 1973). Luborsky et al. 
(1988) also found initial depression to be a positive predictor of 
outcome. It has, however, been difficult to empirically support the 
importance of client motivation due to the vagueness of the construct 
and lack of consistency in definitions across studies. Garfield (1986) 
contended that the empirical evidence concerning the relationship 
between motivation and positive outcome was not particularly 
convincing. 

The relationship between clients' initial expectations of counseling 
and outcome also is not as clear as one might expect. Feste; (1977) and 
Hardin, Subich, and Holvey (1988) found no difference regarding 
precounseling expectations of clients who prematurely terminated versus 
those who appropriately terminated counseling. In their reviews, Lick 
and Bootzin (1975) and Wilkins (1973) both maintained that the 
research on client expectations has been confusing and somewhat 
contradictory, and Garfield (1986) suggested that the importance of 
client expectations as predictors of outcome has yet to be determined. It 
seems that our understanding of client expectation is still at a beginning 
state since the majority of the research has been limited to the 
relationships between demographic variables and expectations (Craig & 
Hennessy, 1989). 

Counselor Characteristics 

Conclusions based on the decades of research into effective counselor 
characteristics remain, for the most part, equivocal (Beuiler, Crago, & 
Arizmendi, 1986). For example, the relationship between counselor age 
and client outcome has little empirical support. Viewing counselor's age 
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as a distinct variable, Beutier et al. (1986) concluded that there was only 
a weak age-outcome relationship. In a well-designed empirical study, 
Robiner and Storandt (1983) found that age similarity between the client 
and counselor did not necessarily improve the counseling relationship or 
the client's satisfaction with the counseling. These findings are 
consistent with Atkinson and Schein's (1986) review on the effect of age 
similarity and their conclusion that it does not account for client 
preferences for counselors or other measures of the counseling process. 

Studies into the impact of counselor experience and training have 
produced surprising results. Stein and Lambert (1984) performed a meta- 
analysis and found that the experience level of the counselor had little 
influence on client outcome. Neither Aueibach and Johnson (1977) nor 
Smith et al. (1980) found experience to be an important influence on 
client outcome. However, Beutier et al. (1986) and Luborsky et al. 
(1988) contended that there is a general trend favoring more experienced 
counselors. Regarding the issue of professional training, Lambert and 
Bergin (1983) reported there is no evidence to suggest that an individual 
from one professional discipline is more effective than another. 
Unfortunately, studies into the variables of counselor experience and 
training have focused on simple correlations between the years of 
counseling practice and/or the type and level of professional training. 
The variable of experience may be better served by investigations into 
the skillfulness of the counselor. 

The effect of the counselor's gender on outcome is more complicated 
than simple differences between male and female counselors. In their 
reviews, Maracek and Johnson (1980) and Tanney and Birk (1976) 
suggested there is neither a clear nor consistent trend in clients' 
preference for the gender of their counselors. Strieker and Shafran 
(1983) contended that an increasing number of studies indicated that 
clients prefer female counselors and stereotypical feminine charac- 
teristics in both male and female counselors. Kirshner, Genack, and 
Hauser's (1979) and Orlinsky and Howard's (1980) outcome studies 
found a trend favoring women counselors, but the experience level of the 
male counselor mediated that difference (i.e., as the male counselor's 
experience increased so did their effectiveness). 

The impact of the counselor's race on the counseling process has 
received increased attention in recent years. Atkinson (1983) and Sattler 
(1977) found support for black clients preferring black counselors, but 
Atkinson (1983; 1985) found little support that other ethnic minorities 
prefer to be seen by ethnically similar counselors. Atkinson, Furlong, 
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and Poston (1986) and Atkinson, Poston, Furlong, and Mercado (1989) 
found that Afro-, Asian-, Mexican-, and Caucasian-American subjects' 
preference for a racially similar counselor was less important than their 
preference for other counselor characteristics (e.g., attitudes and 
personality). Atkinson (1985) indicated that analogue and survey studies 
have generally failed to establish the existence of counselor prejudice 
and stereotyping of clients, but acknowledged that this result may be due 
to existing research strategies. The reviews of Abramowitz and Murray 
(1983), Atkinson (1985), and Atkinson and Schein (1986) suggested that 
the relationship between client-counselor racial matching to outcome 
remains unclear, however, after reviewing 20 years of outcome research, 
Luborsky et al. (1988) concluded that client-counselor racial matching is 
unrelated to outcome. 

A great deal of the work regarding counselor characteristics has 
attempted to identify the ideal counselor personality. However, Beutler 
et al. (1986) maintained that it is unlikely that any single personality 
dimension or client-counselor personality match has a major effect on 
counseling outcome. Beutler (1981) summarized 18 outcome studies and 
found four studies that indicated client-counselor personality similarity 
was related to improvement, while five studies indicated dissimilarity 
was associated with improvement. Atkinson and Schein (1986) 
suggested that for some personality traits client-counselor similarity may 
have a positive impact; whereas, for other traits dissimilarity may 
function to improve compatibility. They also contended that some 
counselor personality traits (e.g., low dogmatism and high vigor) are 
desirable in counseling regardless of clients' personality traits. 

Since counseling is largely an interpersonal activity, an under- 
standable area of interest has been the impact of the counselor's mental 
health on outcome. A major difficulty with the research in this area is a 
lack of consistency in defining mental health. Lambert and Bergin 
(1983) indicated that although the evidence is mainly anecdotal, it 
continues to suggest that the counselor's personality integration and 
adjustment have a positive influence on outcome. A somewhat related 
question to counselor well-being is the impact of personal counseling for 
the counselor and subsequent therapeutic effectiveness. Greenbcrg and 
Staller (1981) cited two studies that found a positive relationship 
between personal counseling and outcome, four studies that indicated no 
difference, and two studies that suggested a negative effect. 
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Summary: Counselor Characteristics 

It is certainly not surprising that there appear to be certain client 
characteristics that influence the outcome of counseling. Generally, the 
research supports a positive relationship between counseling outcome 
and amount of education, intelligence, and possibly the socioeconomic 
status of the client. Neither the gender nor the age of the client appears 
to be related to outcome. The relationship between client race and 
counseling outcome is complex. Although there are difficulties in 
measuring such variables as adjustment and personality, there are 
indications that emotional health and both greater initial depression and 
anxiety are valid predictors of a positive outcome. However, the 
evidence concerning counseling outcome and its relationship to both 
client motivation and client expectations is more equivocal than might 
be expected. 

Although widely studied, research on existing counselor charac- 
teristics is often ambiguous with conflicting findings concerning the 
relationship between counseling outcome and variables such as the 
experience level, age, and gender of the counselor. Numerous reviews 
and research articles addressed the impact of counselor-client gender, 
race, or personality similarity on outcome. Presently these findings are 
unclear. Although the evidence is not usually empirical, there are 
indications that the counselor's mental health may be associated with 
positive results. Clients probably do not respond to single components of 
the counselor but rather to the whole individual, which may partially 
explain the inconclusive findings and suggest that more complex 
research designs may be necessary to explain the characteristics of an 
effective counselor. 

These results concerning existing factors contain implications for 
both counseling practice and training. As can be seen, many of the 
"common sense" client and counselor characteristics, long thought to 
have an impact on counseling, are not significant factors in successful 
outcome. This research could serve as the basis from which practitioners 
could evaluate their beliefs regarding client characteristics. In addition, 
counselor educators could include in the curriculum courses on 
characteristics of clients and counselors that have been identified by 
data-based studies. Such courses would offer student counselors the 
opportunity to dispel their preconceived ideas. 
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The Counseling Relationship 

The counseling relationship has long been acknowledged as an essential 
component of counseling even though there is little consensus 
concerning its definition (Gelso & Carter, 1985). In a systematic review 
of the counseling relationship literature, Sexton & Whiston (1992b) 
defined the relationship as those aspects of the client and counselor and 
their interaction that contribute to a therapeutic working environment 
which in turn may influence client change. Thus, the counseling rela- 
tionship can be viewed as the interpersonal basis upon which technical 
aspects of the session and the implementation of counseling techniques 
are based. 

The empirical literature has supported the claim that the quality of 
the therapeutic relationship is crucial to positive outcome (Luborsky et 
al., 1988; Marziali, Marmar, & Krupnick, 1981). According to Orlinsky 
and Howard (1986), there was a significant relationship between the 
quality of the therapeutic relationship and positive client outcome in up 
to 80% of the studies they reviewed. In a comprehensive review of the 
last 20 years of outcome research, Luborsky et al. (1988) reported that 
88% of the studies reviewed identified a positive relationship to be 
significantly related to client outcome. However, given the vague 
definition of the relationship, questions regarding which specific 
elements of the relationship are related to successful outcome remain. 

Early studies in this area often focused on the importance of the 
"necessary and sufficient conditions" as suggested by Rogers (1957). 
Luborsky et al. (1988) found that the more the counselor conveys 
empathy the greater the likelihood of positive client outcome. In reviews 
of the literature, Mitchell, Bozarth, and Kraft (1977) and Patterson 
(1984) also found support for the effectiveness of the counselor offering 
genuineness, warmth and empathy. The effect of counselor empathy on 
outcome seems particularly strong when measured from the client's 
perspective (Barrett-Lennard, 1962; Martin & Sterne, 1976). 
Additionally, positive regard and warmth toward the client both were 
related to better outcome (Feitel, 1968). 

However, most comprehensive reviews have not found support for 
the importance of all aspects of Rogers' conditions, especially when 
compared to other aspects of the therapeutic relationship. Counselor 
genuineness or congruence had only an occasional, but not consistently 
positive, relationship with successful outcome (Lambert et al., 1986). 
Some findings concerning specific variables varied depending on 
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whether they were measured from the client's perspective, the 
counselor's perspective, or from an independent observer's viewpoint. 
Congruence, which was found to be more strongly related to outcome 
when measured from the client's point of view, was still not as important 
as counselor credibility or engagement in the relationship (Garfield & 
Bergin, 1971; Orlinsky & Howard, 1986). Counselor engagement in the 
relationship (Friedlander, 1981; Jones & Zoppel, 1982) and counselor 
credibility as measured from the client's perspective (Beutler, Johnson, 
Neville, Elkins, & Jobe, 1975; Ryan & Gizynski, 1971) were highly 
related to positive outcome. 

Certain client characteristics also have an impact on the counseling 
relationship. Client involvement in the process and client openness in the 
relationship with the counselor were both shown to be significantly 
related to positive outcome (Kolb, Beutler, Davis, Crago, & Shanfield, 
1985; Lorr & McNair, 1964). In addition, the client's level of warmth 
and acceptance of the counselor have been found to be significant 
factors (Bent, Putman, Kiesler, & Nowicki, 1976; Rosenbaum, 
Friedlander, & Kaplan, 1956). 

There is growing evidence that the reciprocal and interactive quality 
of the therapeutic relationship may be fundamental in understanding the 
ingredients that comprise the therapeutic relationship. The importance of 
reciprocal empathy and affirmation were supported by an overwhelming 
80% of the findings reviewed by Orlinsky and Howard (1986). The 
interactive nature of the relationship also is supported by the findings 
that suggest those counselor variables (e.g., empathy, positive regard) 
were most influential when measured from the client's perspective as 
opposed to global and objective measures (Garfield & Bergin, 1986; 
Orlinsky & Howard, 1986). From an outcome research perspective, an 
effective therapeutic relationship is associated with role investment by 
both client and counselor, mutual affirmation, empathic understanding 
and personal contact, and a strong sense of affiliation and respect 
(Orlinsky & Howard, 1986). 

Given these findings, there has been an increasing interest in models 
of counseling that consider a broad and more interactive perspective of 
the therapeutic relationship. In particular, two of these approaches, the 
therapeutic or working alliance (Bordin, 1979; Stiles et al., 1986) and the 
social influence model (Strong, 1968; Strong & Claibom, 1982), seem to 
have great promise in helping clarify the nature of a therapeutic 
counseling relationship. 
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According to Bordin (1979) t the counseling relationship is best 
considered as a working alliance, which includes: the emotional bond 
between the client and counselor, the quality of the client and counselor 
involvement in the activities of counseling, and the degree of 
concordance between them on the goals of counseling. Although 
originally considered important only in the psychoanalytic literature, the 
concept of a working alliance is now seen as applicable to various 
therapeutic approaches and techniques (Gelso & Carter, 1985). 
Emphasized in this model are contributions of both the counselor and the 
client in an effective working relationship (Horvath & Symonds, 1991). 

The research on the relationship between the working alliance and 
outcome has been generally favorable. Hartley and Strupp (1983) found 
that successful and unsuccessful cases could be differentiated on the 
basis of the strength of the working alliance in the first quarter of 
treatment. Others (Luborsky et al., 1988; Morgan, 1982), reported that 
early indications of the strength of the alliance predicted up to 33% of 
the variance in outcome. Similar results were reported when the notion 
of the working alliance was extended beyond analytic treatments. 
Horvath and Greenberg (1986) reported that as much as 30%-45% of 
the variance in counseling outcome was accounted for by the working 
alliance after only the second or third session. In a recent meta-analytic 
review, Horvath and Symonds (1991) found the working alliance to 
make a significant contribution to counseling outcome. In a review of 
over 100 studies, they reported that a conservative estimate of the 
strength of the working alliance is r=.26. While collaboration, mutuality, 
and engagement were important common elements, the diversity in 
measurements and conceptualization of the working alliance may have 
diminished the reported strength of the relationship with psycho- 
therapeutic outcome (Horvath & Symonds, 1991). 

A second model of the counseling relationship that also focuses on 
client-counselor interaction is the social influence approach (Strong, 
1968). According to this model, the therapeutic relationship is an 
interaction between client and counselor in which the counselor 
develops a position of influential power by enhancing the characteristics 
of expertness, attractiveness, and trustworthiness, thus becoming a 
credible and useful resource to the client. The counselor then uses this 
position of interpersonal influence to assist the client in therapeutic 
changes (Strong, 1968; Strong & Claibom, 1982). However, most of the 
research pertaining to the social influence model has addressed the 
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conditions that may lead to the development of counselor social 
influence (Heppner & Claiborn, 1989). 

LaCrossee (1980) found that perceived expertness, attractiveness, 
and trustworthiness at the completion of counseling were strongly 
related to outcome and accounted for 35% of the variance in outcome as 
measured by goal attainment scaling. Others have found these 
counselor's variables to be related to changes in client self-concept 
(Dorn & Day, 1985) and general satisfaction with counseling (Heppner 
& Heesacker, 1983; McNeill, May, & Lee, 1987; Zamostny, Corrigan, & 
Eggert, 1981). Also, clients who perceived their counselors as being 
more expert, attractive, and trustworthy, as compared to those who 
viewed their counselors less favorably in these areas, were less likely to 
terminate prematurely (Kokotovi & Tracey, 1987; McNeill et al., 1987). 

Heppner and Claibom's (1989) major review of 60 studies on the 
social influence model indicated a variety of counselor verbal and 
nonverbal behaviors affected clients' perceptions of counselors. 
Analogue studies indicated the following: (1) nonverbal behaviors and 
prestigious cues affected perceived expertness; (2) counselor self- 
disclosure and self-involving statements influenced perceptions of 
attractiveness; and, (3) prestigious cues, congruity of roles, responsive 
nonverbal behaviors, positive self statements, and verbal and nonverbal 
cues all were significantly related to perceptions of counselor 
trustworthiness. These analogue studies also indicated that counselor use 
of profanity lowered ratings of attractiveness and that the level of 
intimacy in the counselor's touch was related to the level of perceived 
trustworthiness. Field studies indicated that factors that influence the 
counseling will vary depending on the stage of the counseling. Early in 
counseling, clients' rating of counselor expertness, attractiveness, and 
trustworthiness may be related to the helping role rather than any 
specific counselor behaviors (Heppner & Heesacker, 1982; LaCrosse, 
1980). On the other hand, positive client ratings of counselor 
characteristics at the end of counseling were related to specific counselor 
behaviors (Heppner & Heesacker, 1982). Heppner and Claiborn (1989) 
concluded that the interaction between the client and counselor is not 
static, and in order to understand the relationship it is important to 
consider events at distinct times or stages during the counseling that 
influence this interaction. 
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Summary and Implications 

More than any other element, the counseling relationship is significantly 
related to positive client outcome (Luborsky et al. f 1988). The 
counseling relationship is the foundation from which other counseling 
activities are built and the arena in which interpersonal interaction 
occurs. While certain counselor characteristics such as empathy are 
important, we now know that the counseling relationship is more than a 
set of facilitative conditions. Bordin's (1979) working alliance and 
Strong 's (1968) social influence models broaden the definition of the 
therapeutic relationship and focus on the mutual interactive process 
between the client and counselor. Early outcome results have provided 
strong support for these two models of the relationship. Thus, both the 
working alliance and social influence models may provide important 
perspectives into understanding the active ingredients of a therapeutic 
counseling relationship. 

Some of these research findings can also provide insight into the 
significant elements of reciprocal and interactive counseling 
relationships. Indications are that the relationship should be a personal 
one, where both client and counselor accept and affiliate with each other. 
Given the findings that the client's evaluation of the relationship is more 
strongly related to outcome than other measures, it appears counselors 
need to focus on the client's perceptions. Hence, for practitioners the 
focus should be upon the client's definition of a helpful and engaging 
relationship rather than on elements that have been typically defined by 
the field as "relationship skills." 

To determine what the client perceives as helpful and engaging, 
initial client assessment may include active inquiry into the client history 
of other constructive interpersonal relationships- Similarly, in this 
process the client's expectations regarding the helping relationship 
should be determined. Social influence theory research would support 
the counselor using this initial assessment to enhance the client's 
perception of the counselor being trustworthy, attractive, and expert. 
The working alliance perspective would support using the client's beliefs 
to develop mutual goals and a sense of collaboration. In conclusion, 
these results suggest that the client's perspective of the relationship is 
crucial; however, this does not suggest that counselors should always act 
as clients wish. 

The views and expectations of the client are not the only part of a 
reciprocal or mutual counseling relationship. Since this relationship 
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depends on mutual feelings of respect and regard, counselors may need 
to attend to the personal attributes that contribute to their interactions 
with clients. While research reflected that the relationship should be 
personally involving for the counselor, it doesn't involve characteristics 
of other relationships (e.g., friendship). In fact, according to a social 
influence perspective, the power to influence the client is an essential 
ingredient in a therapeutic counseling relationship. Thus, counselors 
need to achieve a delicate balance between keeping the relationship 
professional, and yet, being personally involved with the client. In 
conclusion, the counseling relationship has been shown to a salient 
factor, yet it is not always easy to establish this critical alliance. 

Counselor education has emphasized human relations skills training 
(Carkhuff, 1980; Ivey, 1971). As Krutz, Marshall, and Banspach (1985) 
suggest, no helping profession has been more deliberate than counseling 
in the development of helping skills training. However, as indicated in 
this review, the counseling relationship is more complex than simply the 
personal connection between client and counselor promoted by the 
helping skills movement. The counseling relationship is a mutual 
interactive process between the client and counselor rather than a simple 
demonstration of counselor skills. Based upon these findings, curricular 
attention should focus on relationship models supported by research 
(working alliance, social influence) and focus less on those skills that 
appear to be less effective (facilitative conditions). Martin (1990) aigued 
that the field can now move beyond psychological skill training into a 
more comprehensive educational experience. The comprehensive 
educational experience suggested by Martin may be enhanced by 
focusing curricula on the elements of the interactional process between 
the client and counselor, rather than exclusively on the counselor's 
implementation of helping skills. 

Counseling Session Factors 

Within the context of counseling, the practitioner must make numerous 
organizational and logistical decisions that have the potential to 
influence counseling outcome. Session factors include, among many 
others, how long counseling will last, which modality to use, how many 
and how often to schedule sessions, and what fee to charge. Theoretical 
models differ on the importance attributed to these decisions. Do these 
session factors influence the outcome of counseling? 



100 



96 



Counselor Efficacy 



While some of these factors have been shown to significantly affect 
client outcome, the research indicated that some generally expected 
session factors make little difference. Three oiganizational aspects of the 
counseling session seemed to make little difference to the eventual 
outcome. Whether counseling occurred in group or individual settings 
(Schmidt, 1982), whether it occurred once a week as opposed to more or 
less frequent sessions, or whether the client paid a fee (Pope, Geller, & 
Wilkinson, 1975) did not significantly affect client change. The lack of 
an association between fee and outcome may be somewhat surprising. 
The purpose of much of the research into the influence of fee payment 
has been to investigate the role of client investment. It is becoming 
increasingly clear that monetary remuneration may be a poor indication 
of the more global concept of client investment 

The timing and procedures of the initial stages of counseling have 
also been investigated. In their recent review of the literature, Luborsky 
et al. (1988) found that a long wait between applying for and beginning 
counseling was negatively related to outcome. On a similar note, Peiser 
(1982) found that clients who missed one of their first four scheduled 
sessions are significantly more likely to have negative outcomes than 
those who kept those initial appointments. The manner in which the 
client and counselor initially structure their roles was also important. 
Outcome was enhanced when a more collaborative rather than 
authoritarian style was initiated (Cooley & Lajoy, 1980; Rudy, 1983). 
Similarly, clients who were prepared for their roles either prior to or as a 
part of the initial stages of counseling, had more positive outcomes than 
those who were not (Eisenbeig, 1981; Friedlander, 1981). Clients with 
low initial prognosis seemed especially to benefit from being prepared 
for their role in the therapeutic process (Jacobs, Charles, Weinstein, & 
Mann, 1972). 

Are more sessions bettert A number of studies have focused on the 
relationship between the number of sessions and client outcome. 
Luborsky et al. (1988) found that the number of sessions was positively 
related to greater client improvement. On the other hand, Smith et al. 
(1980) identified a curvilinear relationship between the number of 
sessions and outcome. In one of the most complete reports, Howard, 
Kopta, Krause, and Orlinsky (1986) also found a curvilinear relationship 
in which 50% of the improvement occurred by the eighth session, 75% 
occurred by the end of 26 sessions (once a week therapy over 6 months), 
and 85% of improvement occurred by the end of the first year of weekly 
counseling. 
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Summary and Implications 

On the basis of these findings it seems that certain organizational 
decisions made by practitioners regarding session factors do affect 
counseling outcome. For example, counseling should quickly follow 
initial contact by the client. The client's attendance and involvement in 
the first sessions appears to be critical, suggesting that counselors 
attempt to enhance those factors that may help facilitate early 
attendance. In addition, the manner in which counselors prepare clients 
for their role is also important to the final outcome. Furthermore, it does 
not seem to matter if clients are seen as individuals or in groups, which 
may be a particularly relevant finding to those counselors in settings that 
often use group approaches. Also, it may be useful to look at methods 
for encouraging client investment and realize that requiring clients to 
pay a nominal fee may not be the most effective way to encourage an 
investment in the counseling process. 

Along this line, some practitioners may want to examine their 
conceptions concerning the time frames within which they expect client 
changes to occur. While there are certainly conditions, clients, or 
situations that require long-term counseling, on the average, the majority 
of client change occurs within the first 6 months of weekly counseling. 
Consequently, counselors should expect to be most influential and 
therapeutic in the early stages of counseling. Thus, it is incumbent on 
counselors to promptly initiate work rather than losing precious time in 
the early stages. Attending to important elements in the counseling 
relationship, such as social influence variables of expertness, 
attractiveness and trustworthiness may also enhance the ability to impact 
a client early in counseling. Counselors must also be adept at identifying 
progress within counseling and at quickly rectifying the situation when 
the counseling process has stalled. 

Specific Counseling Approaches and Techniques 

Historically, a major concern in outcome research has been the attempt 
to identify which technique or model of counseling is most useful for 
which client and under which conditions (Paul, 1967). This question 
about which approach is "best" has spawned a number of comparative 
outcome studies (Luborsky, Singer, & Luborsky, 1975; Smith et al., 
1980). To date, the outcome research has been unable to identify a more 
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favorable relationship between any one of the over 130 different 
"schools" or specific counseling approaches and counseling outcome 
(Garfield & Bergin, 1986; Lambert, 1991; Smith et al. v 1980). In 
comparing psychotherapeutic approaches, Luborsky et al. (1975) found 
that the various therapeutic approaches were equally effective and thus 
proclaimed, using the words of the Dodo bird in Alice in Wonderland, 
that "Everyone has won and all must have prizes." Stiles, Shapiro, and 
Elliott (1986) concluded that, based on a substantial body of evidence, 
one could reach the conclusion that the outcomes produced by different 
therapeutic approaches are equivalent. However, Kazdin (1986), 
suggested that these cumulative findings were at best mixed and difficult 
to assess. More recently, Beutler (1991) has suggested that conclusions 
of comparability contribute to the "uniformity myth" and thus confound 
the potentially useful process of prescriptive matching. 

The questions regarding which approaches are most effective are 
both theoretically and methodologically complex (Stiles et al., 1986). 
For example, in regard to specifically targeted client problems such as 
phobic, anxiety, and skill deficit problems, numerous researchers have 
found that a small but consistent advantage exists for cognitive and 
behavioral methods over traditional verbal and relationship therapies 
(Dush et al., 1983; Lambert et aL, 1986; Miller & Bennan, 1983; 
Nicholson & Berman, 1983; Shapiro and Shapiro, 1982). Conversely, 
other studies have supported Luborsky et al. (1975) and found little 
advantage for behavioral over non-behavioral treatment methods (Klein, 
Zitrin, Woemer, & Ross, 1983; Stiles et al., 1986). 

In one of the most recent and methodologically sound comparative 
studies, the National Institute of Mental Health collaborative research 
program studying the treatment of depression, no differences were found 
tetween cognitive behavior therapy and interpersonal counseling (Elkin 
et al., 1989). These conclusions compare favorably with those of a recent 
study that explored the differences between exploratory-insight-oriented 
(EIO) versus reality-adaptive-supportive (RAS) approaches in working 
with schizophrenic patients. In this study, Frank and Gunderson (1990) 
found that although patients in both groups made substantial 
improvements, there were no differences between the approaches in 
client outcomes. They noted that since the outcomes were not related to 
differences in approaches, they have refocused their attention on 
identifying the "common factors" that explain client change. 

These results have produced an interesting paradox for the 
counseling practitioner. On the one hand, the empirical research has 
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suggested that there is no difference between diverse therapeutic 
approaches. On the other hand, the process and content of various 
approaches appears to be dramatically different. A number of important 
issues require further consideration in order to understand this apparent 
paradox. First, it may be that differences among approaches do exist but 
are being missed due to research methodology problems (Kazdin, 1986; 
Stiles et al., 1986). For example, many times comparative studies use 
imprecise outcome measures and research designs (Kazdin, 1986; Stiles 
et al., 1986) or may treat clients, therapies and methods as inter- 
changeable (Kiesler, 1966) thereby averaging any effect of treatment 
across heterogeneous clients, counselors and settings. Persons (1991) 
noted that contemporary outcome studies have not adequately measured 
the theories being compared in that they do not provide individualized 
assessment or treatment planning. Instead most studies standardize all 
approaches to clients, potentially obscuring differences between 
theoretical approaches. Similarly, the research concerning differences in 
theoretical approaches is often disseminated in research reviews, where 
there are other methodological problems that may obscure variations 
among the approaches. Box score and narrative methods, used in many 
of these reviews, may provide an imprecise comparison of the 
differences in this complex question. However, as suggested by Kazdin 
(1986), the issue of treatment comparability is very complex and cannot 
be easily dismissed by the call for improved research design. 

A second explanation may be that there are no important differences 
among counseling approaches. Given this explanation, one could aqgue 
since counseling is effective, any degree of client change must be due to 
common processes shared by all approaches (Garfield & Beipn, 1986; 
Gelso & Carter, 1985; Goldfried, 1980; Highlen & Hill, 1984). While a 
"common factors" explanation has been previously proposed, to date the 
outcome literature has not specifically identified the germane core 
elements of counseling that influence outcome. In a promising move, 
researchers have begun to explore the interactive aspects, such as the 
therapeutic alliance between counselor and client (Gelso & Carter, 1985; 
Horvath & Greenberg, 1986) and the social influence model (Strong & 
Claiborn, 19&2). This interactive focus may help explain specific 
elements of these common factors. 
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Specific Techniques that Result in Client Changes 

Specific techniques used to elicit client changes have also been a 
common focus of outcome research. But do any of these common 
methods have a positive impact on counseling outcome? A number of 
frequently used techniques received either mixed or minimal empirical 
support. For example, exploratory statements by the counselor intended 
to clarify meanings or experiences were not found to be consistently 
related to positive outcome (Orlinsky & Howard, 1986). While Elliott, 
Baker, Caskey, and Pistrang (1982) found that giving advice was 
positively related to counselor-rated client outcome, most other reviews 
have found little support for giving advice and being directive (Mintz, 
Luborsky, & Auerbach, 1971). Orlinsky and Howard (1986) found that 
reflections or restatements to check the accuracy of the counselor's 
understanding of the client and/or to facilitate future self-explorations 
were neither beneficial nor detrimental. When clients were directed to 
attend and focus on their affective states, it was sometimes but not 
always helpful (Nichols, 1974). Little was accomplished when 
counselors used immediacy and focused clients on here-and-now events 
in counseling; furthermore, immediacy under conditions of low empathy 
and genuineness may even have been harmful (Orlinsky & Howard, 
1986). Finally, counselor self-disclosure was only occasionally found to 
be helpful (Elliott et al., 1982; Hayward, 1974; Watkins, 1990). 

Confrontation was often but not consistently found to be related to 
successful outcome (Greenberg & Rice, 1981). Directing and focusing 
the client's attention as a form of confrontation was sometimes important 
(Nichols, 1974); however, not all focusing was of equal importance. 
Orlinsky and Howard (1986) suggested that outcome was enhanced 
when c ./iselors directed the conversation to the client's presenting 
problems. In fact, when clients focused more intently on problematic 
issues they did substantially better in counseling. The use of questions 
to explore aspects of the client's presenting concerns rather than to 
gather information was found to be frequently helpful (O'Malley, Suh, & 
Strupp, 1983). Finally, encouraging client's independence also seemed to 
be important (Luborsky et al., 1988). 

While a variety of counseling techniques contribute to client change, 
the most important aspect of any counseling technique may be the 
skillfulness with which these interventions are used and implemented 
(Feifel & Eells, 1963; Nash, Hochn-Saric, Batth, Stone, Imber, & Frank, 
1965; Sachs, 1983). Sloane et al. (1975) suggested that the skillfulness 
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of the counselor was important in more than 70% of the cases they 
reviewed, while Luborsky et al. (1988) found all reported studies 
strongly support the importance of skillfulness. Lambert (1989) 
suggested that the ability of the individual counselor can have a major 
impact on the process and outcome that exceeds the impact made by any 
technique ancVor approach. 

Summary and Implications 

Which method or technique is best for which client and under what 
conditions? Kazdin (1986) suggested that while the intent of 
comparative outcome studies and major literature reviews has been to 
resolve the paradox and answer these questions, the results have 
produced unclear conclusions that may have fueled the controversy. 
While a number of claims have been made about the equivalence of 
approaches in counseling, these claims appear to be somewhat premature 
given the theoretical and methodological complexity of the issues 
surrounding comparative outcome studies and the current status of the 
empirical data. 

However, to date, the research indicates that no one theoretical 
approach to counseling is any more effective than any other. Similarly, 
adherence to any one theory or approach does not guarantee successful 
outcome. However, these results do hot necessarily indicate that theory 
is irrelevant Cormier and Hackney (1987) suggested that a theory may 
be a useful conceptual tool in helping practitioners organize and direct 
their activities in a systematic fashion. Highlen and Hill (1984) 
interpreted these results to indicate that the field should woik toward 
convergence and identification of elements that facilitate client change 
that are common to many psychotherapeutic approaches. Garfield & 
Bergin (1986), Frank (1974), and Sloane et al. (1975) have also 
proposed that the field concentrate on the "common factors" that are the 
underpinnings of therapeutic effectiveness. The role of theory on 
outcome will continue to be debated and with advances in method- 
ological approaches a better understanding may emerge. 

It is also difficult to evaluate the contribution of specific techniques 
outside the context of other elements within counseling, but research 
indicated that some specific techniques contribute to positive outcome. 
Most techniques that help clients focus on their presenting concern seem 
helpful. Certainly there is no "magic" in any technique, and in fact it 
may be the method and skill of the counselor using the intervention with 
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the client that is most important. The choice of therapeutic technique 
depends on the quality of therapeutic relationship, the structure of the 
session, and the goals of the client. Each technique must be evaluated for 
the individual client and the support from the empirical literature. 



Client Experiences 

Clients are asked by counselors to engage in numerous experiences 
throughout the course of counseling. Sometimes they are asked to self 
disclose, sometimes to problem solve, or maybe to express their feelings 
or develop insight. Are these experiences and activities related to 
successful counseling? 

Several common experiences clients undergo in counseling that have 
an impact on outcome have been empirically investigated. Active client 
involvement in counseling was found to be very important to positive 
client outcome. In particular, outcome seems to be enhanced when 
clients develop a problem-solving attitude (O'Malley et al., 1983; 
Luborsky et al., 1988) and become actively engaged in experiences that 
help them master problematic situations (Luborsky et al., 1988). 
Schauble and Pierce (1974) found that involvement is augmented when 
clients attribute their problems to themselves as opposed to outside 
events. The development and maintenance of client expectations for 
positive change in counseling were related to positive outcome 
(Luborsky et al., 1988). In addition, there seemed to be support for a 
trend indicating clients experiencing a greater ^ange of emotions had 
more positive outcomes than those who experience! only a narrow range 
(Luboreky et al., 1988; Nixon, 1982). Finally, a process that encourages 
the clients to talk was related to positive client outcome (Sloane et al., 
1975). 

Three client experiences commonly assumed to play important 
role in counseling were not supported by the empirical research Neither 
client insight nor client self-exploration were related to successful 
outcome (Morgan et al., 1982). Similarly, the importance of client 
motivation received only mixed support and was strongest whsn 
measured from the counselor's perspective (Keithly, Samples, & Strupp, 
1980;Prager, 1971). 

On the surface, these results may seem difficult to understand given 
the present values and theoretical traditions of the counseling profession. 
However, these findings may make sense given a different view of the 
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communication process in counseling. Consider, for example, the social 
influence model of counseling, which has generated interest in the 
impact of message variables on the client's counseling experience. In 
particular, Claibom (1982) and others (Levy, 1963; Strong & Claiborn, 
1982) suggested that, regardless of content, it is the discrepancy of the 
information provided by the counselor that results in client attitude 
change. While still tentative, these findings suggested that clients who 
receive information that is moderately discrepant from their own beliefs 
may be more likely to change their beliefs than those who experience 
either high or low discrepancies (Heppner & Claibom, 1989; Strong & 
Claiborn, 1982). This discrepant information can be presented to the 
client using techniques such as counselor interpretation, reflection, or 
reframing. Thus, insight that does not provide any new and different 
information may not be helpful. 

Summary and Implications 

The experiences clients engage in during counseling do have an 
important impact on clients' success or failure in reaching their goals. 
Understanding which experiences are most helpful and in what manner 
they may facilitate change is complex. Presently, it appears that client 
experiences can be most productive when a skilled counselor facilitates 
client involvement and when the client develops a problem-solving 
attitude and engages in mastery behaviors related to germane situations. 
Furthermore, it seems counselors need to encourage clients to have a 
broad ran 0 e of emotions and they need to promote an expectation of 
positive change. 

One conclusion that can be gleaned from these data may be that 
practitioners and educators should reconsider the extent to which the 
experiences they promote in counseling are related to outcome. For 
example, client self-exploration and insight are often considered to be 
important steps for clients to take in the resolution of their concerns. 
Yet, studies have indicated that self-exploration and insight are not 
necessarily related to outcome unless specifically focused on the client's 
presenting problem. Luborsky et al. (1988) contended that it was 
astonishing that there was not more support for the concept of insight 
considering its emphasis in most approaches. In light of the research, it 
is perplexing that so much time is spent on insight and self-exploration 
when problem solving and mastery are more significantly related to 
successful outcome. For the practitioner, these findings would suggest 
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that insight and exploration be purposefully related to understanding 
elements of the problem currently facing the client 

Implications and Applications 

This chapter has provided an overview of current trends in counseling 
outcome research. The voluminous results of studies conducted to 
investigate counseling process strongly suggest that counseling can be 
effective in helping clients make positive changes. What is not so clear 
are those characteristics of counselor and client, aspects of the 
counseling relationship, session factors, or models and techniques that 
may contribute to counseling success. In fact, no single counselor or 
client characteristic or combination of client or counselor characteristics 
can account for successful outcome. No particular theoretical modei or 
magical technique can independently ensure success. The quality of the 
relationship was consistently noted as a significant factor in outcome. 
However, the significance of other seemingly unrelated elements of the 
counseling process leads to the conclusion that successful outcome 
results from more than just a quality client/counselor relationship. One 
can only conclude that counseling is complex with multiple factors that 
combine to influence successful outcome. 

The preponderance of outcome evidence also leads to the conclusion 
that counseling success is not at all guaranteed. In fact, as many as 10% 
of clients may actually get worse as the result of participating in 
counseling. While a number of factors contribute to negative results, the 
implications of these results support the contention that negative results 
often are related to the counselor. The conclusion: Some counselors are 
better at producing positive outcome than others. Along this line, 
Lambert (1989) argued that the individual counselor is a significant 
factor in outcome. Thus when counseling concludes with less than 
favorable results, it is the counselor who must take the responsibility to 
alter his or her approach and interventions. These findings support the 
position espoused by Haley (1976) that the counselor is responsible for 
the results of therapy. In addition, an unskillful counselor cannot be 
ignored by colleagues and supervisors because "no harm can be done." 

The present status of the outcome research also leads to the 
conclusion that many questions regarding the specific elements that 
contribute to successful counseling are, as of yet, unanswered. In order 
to adequately answer these questions, theoretical and empirical inquiry 
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into counseling efficacy should refocus. Instead of searching to find 
which approach is best, it may be most beneficial to investigate what is 
common among successful counselors regardless of their theory or 
model. In fact, it can be argued that a "common factors" model is 
presently supported by the counseling outcome literature. Lambert 
(1986) suggested that these common factors account for 30% of change 
while techniques may account for 15%, expectancy (placebo) effects 
another 15%, and spontaneous remission 40%. Unfortunately, what the 
research does not provide is a clear indication of those specific 
"common factors" that are shared by all successful counseling. 

Along this line, it seems that a multidimensional model may be 
necessary to successfully account for the "common factors." 
Atheoretical factors, such as those comprising the model used to 
organize this review (see Figure 1), can be supported by the counseling 
outcome research and may provide an initial step toward understanding 
the nature of these "common factors." For example, characteristics 
shared by both clients and counselors influence the initiation of the 
counseling relationship. The type of relationship sets the foundation for 
various session factors. These session factors set the parameters for the 
conduct of counseling. Within those parameters, various techniques may 
help clients focus on their presenting concerns. As a result of these 
techniques and the relationship with the counselor, the client has certain 
experiences. Each of these different dimensions of the counseling 
process uniquely contributes to successful outcome. Future counseling 
outcome research should employ techniques that investigate the 
interconnections among these various elements, rather than viewing 
counseling as a series of unrelated factors. 

Although controversial, the outcome research could prove very 
important in improving counseling practice. For example, Paul (1967) 
suggested that one of the primary goals of a counselor is to be able to 
discern which general strategies and specific techniques are appropriate 
for which client, under which circumstances. But what is the basis upon 
which those decisions are made? Unfortunately, there are indications that 
counselors are not exempt from negative or stereotypical attitudes 
toward females (Hill, 1975), males (Robertson & Fitzgerald, 1990), 
minorities (Wampold, Casas, & Atkinson, 1981), and the aged (Myers, 
1988). In fact, many client and counselor characteristics commonly 
assumed to be important factors influencing effective counseling do not 
seem to be supported by the empirical literature. Holding true to beliefs 
without empirical support may inadvertently lead counselors to intervene 
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with clients in ways that are not helpful. One of the great values of the 
outcome literature is that it has the potential to serve as a reliable basis 
for clarification of these beliefs. 

Using the outcome literature as a foundation for training models 
could also provide the basis for important curricular changes. For 
example, one of the hallmarks of many counselor education programs is 
the counseling theory class. This class is often seen as providing models 
from which student counselors will operate with their clients. However, 
there seems to be little evidence that adhering to a theoretical model is 
related to client change. Counseling theories may assist counselors in 
organizing information about client problems and interventions (Cormier 
& Hackney, 1987) but do not substitute for an understanding of the 
elements of the counseling process and the ability to implement 
appropriate strategies to best meet the needs of clients. An empirically- 
based curriculum would more likely include course work and 
experiences that focus on the "common factors" of successful 
counseling. Counseling theory and counseling outcome research could 
actually go hand-in-hand. Knowledge of the outcome research could 
serve as the knowledge base from which future counselors become 
critical, informed consumers of counseling theory and technique. 

The minimal impact of outcome research on practice and training is 
certainly interesting given the great wealth of available research studies. 
However, this situation is understandable when you consider that many 
counseling practitioners and counselor educators will find numerous 
results within these pages that do not match their personal beliefs about 
effective counseling. For example, that insight is not "curative" or that 
all approaches are equal may be difficult to accept Unfortunately, these 
apparent discrepancies between research and practice often result in the 
dismissal of research findings as irrelevant Indeed, numerous questions 
have been raised about the current counseling research methodologies 
(Kazdin, 1986; Persons, 1991; Lipsey, 1990). These methodological 
issues are beyond the limits of this presentation. However, even given 
these methodological questions, the outcome research findings cannot be 
dismissed as mere methodological error. More comprehensive 
methodologies, such as meta-analytic approaches aftd quality clinical 
trials, appear to increase design sensitivity (Lipsey, 1990) and thus 
improve measures of counseling effectiveness. In addition, the consistent 
trends that appear in quantitative and qualitative reviews and clinical 
trial studies only lend support for the value of outcome research as a 
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source from which to evaluate our understanding and practice of 
counseling. 

There is also an ethical dimension to the question of what basis to use 
to make decisions in counseling and training that is often overlooked. 
Counseling carries a substantial ethical responsibility to make the 
welfare of the client the highest priority. The ethical code of the 
American Counseling Association (AACD, 1988), as well as of other 
professional organizations, clearly states that practitioners should use the 
research to guide their practice (AAMFT, 1988; APA, 1989). Lambert 
(1991) has argued that a counselor cannot meet these ethical standards 
without a thorough knowledge of the empirical literature. The 
counseling outcome research continually provides significant 
information and counselors need to remain current concerning changes 
within their field allowing them to provide the best care for clients. 
Counselor training programs influence fiiture counselors and, thus, have 
a similar professional and ethical responsibility to examine all the 
evidence when making curricular decisions. The outcome research can 
be an important aid to both practitioners and educators. 
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Chapter 8 



Important Considerations in 
Disseminating Counseling 
Outcomes Research 

Garry R. Walz 

Knowledge keeps no better than fish. 

Alfred N. Whitehead 



Introduction 

In a real sense a focus on counseling outcomes should employ the dual 
perspective of both research data generator and research outcomes 
user. A focus on either one to the exclusion of the other may very likely 
fail to enhance counseling efficacy. A near exclusive focus on research 
design and statistical treatment can result in counseling outcome 
research studies that are rigorous but of only limited utility to those who 
profit most from their use. Conversely, an almost exclusive focus on the 
specific questions and interests of a user/practitioner while ignoring 
important design and analysis considerations can lead to research that 
targets important topics but probes the many dimensions of those topics 
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poorly. For cither practicing counselors who need valid and viable 
information for enhancing their counseling efficacy, or researchers who 
have a commitment to program research which has professional impact, 
both emphases are important. 

Ten Principles of Counseling Outcome Research Results 

In this chapter I have endeavored to articulate ten principles of 
counseling outcome research results which apply to both generators and 
users of research. I think of the principles as stimuli to further 
exploration by a reader on the ideas they present and the questions they 
raise. We believe that to think of counseling outcomes research as only a 
body of knowledge with little consideration of how and why it was 
generated and for whom it is intended, and how it can be most 
expeditiously disseminated to those who can benefit most from its use, is 
to settle for functional trivia rather than potency. Counseling efficacy 
and advocacy go hand in hand. Hopefully, these principles will help 
counselors to reflect and act upon both in a concurrent and congruent 
fashion. 

/. All research reports should clearly specify in an initial statement 
who was involved in the framing of the objectives of the research and for 
whom the research is intended. Both practitioners and researchers who 
arc now regularly engaged in the retrieval of research studies pertinent to 
their interests have an advantage, not accorded them some years ago, of 
being able to search the available body of literature by specifying that 
they arc interested only in retrieving research studies and in some cases 
specifying the type of research about which they wish to retrieve 
information. 

Thus, for retrieving information on subjects which may potentially 
yield enormous numbers of studies frequently numbering in the 
thousands, a person is able to winnow the number down to a useable 
quantity. We believe that this is an important principle for two major 
reasons. First, it greatly aids the person engaged in the search aspect of 
research, that is retrieving information relevant to his/her interest and 
thus avoiding needless screening and assessing of laige numbers of 
essentially irrelevant studies. Second, it prods a given researcher and 
writer to be clear in his/her own mind as to the audience for whom the 
research is intended. Researchers cannot be all things to all counselors! 
The goals which fashion a research endeavor and the manner in which it 
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* cheated are both instrumental in determining for whom it will 
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has the potential for becoming a publication that covers issues and topics 
of interest to a wider public. This could lead to the reporting of 
important counseling outcomes research in the public media to the real 
benefit of the counseling community. 

An excellent source for the dissemination of ideas, either about major 
professional issues or specific research or theoretical topics, is the use of 
the ERIC Digest. A single page, two-sided report of about 1500 words, 
the Digest is a carefully targeted information analysis product which in a 
very succinct and authoritative manner provides the reader with highly 
relevant and up-to-date information on a given topic. Particularly with 
the advent and proliferation of electronic communication, all counselors 
may wish to consider the use of the digest format for providing 
information to different publics in a format that conserves both the 
writer's as well as the user's time and energy. 

3. Front-end framing which reflects significant stakeholder interests 
is an essential element of outcome-focused research. It is not unusual for 
counselors to discuss whether or not they can use the results of a 
particular study. Typically, the discussion is somewhat tortuous because 
the "real questions" that different persons in the group have are not 
considered or spoken to in the research report. 

It is clear that the partiality of many major funding bodies is to 
earmark their funds for research and development by professional 
development institutes. In these institutes the focus and agenda building 
is determined by a mix of educational specialists with particular 
attention given to the needs of those who may potentially benefit from 
the research. Being sure that those whom you wish to use your results 
have early opportunities to share their needs and interests is to provide a 
greater assurance that there is more return for the "buck" than if 
questions of application do not occur until after the research has been 
completed. The clear principle here is that stakeholders, with particular 
reference to those for whom there is an expectation that they will use the 
results, should have significant input in the early discussion and design 
stage of research endeavors. It is possible, even probable, that if we were 
to involve practicing counselors more in research on client outcomes that 
we would obtain information which could be used by counselors to 
enhance the role they play in the counseling process and thereby extend 
counseling efficacy. 

There are key phrases for specialists in many fields that represent the 
wisdom of experience. Thus, in real estate it is said that what is 
important in the purchase of property is "location, location, and 
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location." Perhaps the admonition for counseling outcomes 
investigations should be "client front-end framing!" 

4. The dissemination of results and findings growing out of 
counseling outcome investigations should be in a format appropriate for 
the intended end user. From a purely self-interested standpoint, a 
researcher should have as a major objective, "How can I communicate 
with my intended audiences in a way that will maximize the ease with 
which they understand and apply the results of my work." Operating 
from this perspective, the researcher is likely to maximize the benefits of 
his/her own efforts and to reap the rewards and satisfactions of having 
been influential and helpful to others. To approach the reporting of 
counseling outcome results in this fashion calls for imagination and 
flexibility. Our continued interactions with practitioners has led us to 
believe that the "what woiks" focus is a primary concern and interest of 
hard-pressed practitioners. They are on a never-ending quest for 
information about programs and practices with the clear potential to 
bring about significant improvements over existing programs and 
practices. Clear guidelines for how these practices may be adopted and 
implemented are pure gold for a hard-pressed practitioner. It also has 
been found that "peer reporting" on the use of a particular innovation (or 
the use of case studies and anecdotal descriptions of someone's use of a 
new procedure or practice) is a meaningful and interesting way to 
present research outcome findings. One need not labor the point that 
paragraphs of esoteric narrative are not nearly as interesting or useful to 
someone whose greatest need is for applications — rather than for 
writings — which focus on a specific effort to bring about an 
improvement in a counseling program or practice. How a new practice 
works and what has been learned from the efforts of others about 
applying and using counseling outcome results are important concerns to 
counseling researchers. 

The format which has been adopted by ERIC/CAPS and has proven 
to be useful is that presented in the Crime de la Crime counseling series. 
In a Crime de la Crime book, the writer first of all provides a solid 
rationale and conceptual base for his/her counseling approach. This is 
then followed by a series of discussions by users in different locations 
and settings about: (1) what they did, (2) the nature of the setting in 
which they did it, (3) the way in which they introduced the innovation, 
(4) the outcome that resulted, (5) what they learned from the experience, 
and (6) what they recommend to others thinking about adopting their 
innovative program. 
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Persons using the Crfeme de la Crfeme monographs have reported that 
the multiple descriptions of applications and use prove particularly 
rewarding because a reader can pull from the different descriptions 
relevant and motivating comments and ideas. This gives the readers the 
opportunity to develop their own syntheses and to decide how they can 
integrate the experiences of a number of users in a way that is applicable 
to their own setting. This also provides a sense of "ownership" of the 
innovation. This ownership leads also to a greater motivation to see the 
innovations succeed: Stakeholders will devote their energies to 
implementing the innovations because they represent their own thinking 
and energy as well as the thinking and energy of the originator of the 
innovations. 

5. Counseling efficacy research should include studies which provide 
confirmation of counseling programs and practices as well as 
exploratory studies which probe the efficacy of new, untried counseling 
interventions. A time honored and highly creditable form of research in 
both the physical and behavioral sciences is confirmatory research (or 
the non-confirmation) of existing theories and beliefs. A one-time 
finding of the relationship between dynamic variables with a narrow 
unrepresentative population sample, e.g., college sophomores, is hardly 
cause for jubilation or a belief that the connection has been 
demonstrated. The one-time validation leading to an assumption that it 
has become a principle of counseling may not only be in error in itself (a 
statistical or research artifact not demonstrable in further studies) but can 
lead to a chain of fallacious research based on the assumption that the 
original connection or outcome was validly demonstrated when in fact it 
cannot be replicated in subsequent studies. 

Undoubtedly there is less glamour or excitement associated with 
confirmatory research. In most cases one is not investigating an exotic or 
exciting new idea. Rather, one is looking into whether the questions 
posed by exploratory research have substance and validity. It is, we 
believe, a highly worthy pastime and one that needs greater recognition 
and emphasis as an important research endeavor. It would seem that 
courses in research for counselors could well involve projects where 
students as a team have as their goal the replication of a major 
counseling outcome (or a crucial aspect of that outcome) as an early 
lesson in research design. This approach would not only help them to 
understand the modus operandi for confirmatory research and 
replication, but would help them to develop an attitude that emphasizes 
the need for replication before an outcome is accepted or acted upon. 
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6. We need more attention to and discussion of the ethics and 
problems associated with secondary analysis. Among doctoral students 
there is a great press to find a topic for their dissertation, gather their 5 ^ 
data, and get on with analysis and writing. The advantages of using data 
from a completed study — frequently of proportions far greater than 
erstwhile doctoral candidates would accomplish with their own 
resources — are highly attractive. A quick entry into data analysis, the 
prestige of being associated with a major study, albeit after the fact, and 
a kind of insurance that their work will be sans major obstacles or 
disruptions, makes secondary analysis an attractive research design. 
Attractive indeed if the goal is to do a creditable research job and get on 
to the transition from student to faculty or professional practitioner. Nor 
is the appeal confined to doctoral students. Certainly, many faculty 
members are attracted to research opportunities in secondary analysis 
where start-up demands are minimized. 

As attractive as secondary analysis can be to students and faculty 
members alike, we are struck by how little discussion has been devoted 
to the topic. We believe two major types of issues need to be discussed. 
First, is the question of whether this piggyback research provides an 
adequate opportunity for fledgling researchers to learn the challenges 
and demands of initiating research. Secondary analysis is frequently neat 
and clean but is it adequate preparation for undertaking initiatory 
research? A review of sample doctoral dissertations suggests that the 
secondary analysis route is indeed highly attractive to doctoral students. 
But are they possibly losing out on the opportunity to acquire effective 
research skills by being efficient in conducting their doctoral research? 
A second question, ethical rather than procedural in nature, concerns 
who deserves "first billing" or how to assign the credit for the research. 
Does credit for all subsequent research on a major research project 
legitimately belong to the original researchers and is hence citable by 
them? How much credit is to be given to the secondary researchers? We 
believe questions like these are deserving of more attention and 
discussion if we are to more adequately explicate the use of counseling 
outcomes research. 

7. A relatively undiscussed question is "Where is counseling research 
initiated?" and "Who are the initiators?" Answers to these questions 
play a very important role in determining the utility as well as the use of 
counseling outcome research. These questions may be the ones to ask: 
"What is the nature of the research enterprise in counseling?" "Who 
docs this research and where are they located?" The answers to these 
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questions may lead to understanding the characteristics of the counseling 
research being generated. It has been noted by those who do comparative 
studies of education and educational research that, in contrast to 
educational research in many foreign countries (e.g., Denmark and 
Holland) where practitioners are virtually involved in all phases of 
educational research, it is the American practice for researchers to be 
university or research center staff for whom research is a major, if not 
their only, priority Practitioners are frequently involved only at the very 
end when they are presented with the research findings. An alternative 
model is one where researchers and practitioners rotate roles and 
variously participate in both research and the delivery of services. As 
suggested, this has become a more common model for European 
educational research. 

Basic to this whole discussion is how research results are best 
assimilated and used. The superordinate/subordinate model perceives 
end users, e.g., practicing counselors and teachers, as persons who are 
given research results so they may use them to improve their practices. 
Essentially, under this model researchers fill empty vessels. The 
researcher decides what is is important to research and after completing 
the research decides what to use and how to disseminate the results. 
Under the alternative approach to dissemination, the stakeholders are 
partners in their search process from the initial framing of the questions 
to be researched to interaction on and consensus decision making about 
what the research results means, how they can be used and to whom and 
in what form they should be disseminated. Many professional 
educational development institutes and programs are located in 
intermediate school districts and associated with a university but not 
controlled or directed by university staff. Rather than fill an empty 
vessel, this approach more fittingly could be described as consensual 
decision making about what vessel to use and what to put in it. 

The bottom-line question for counseling is how counseling research 
is viewed by those who most stand to benefit from it Are the potential 
consumers and users of counseling research both attracted to it and 
effective users of it? If we cannot answer yes to the aforementioned 
questions then it would seem we do need a new model for the conduct 
and dissemination of counseling outcome research. 

8. We need attention to the process by which a counselor confronts 
new knowledge and makes appropriate changes in his/her beliefs and 
behavior. As researchers it would often seem that for many the final act 
in the research process is the writing of the journal article describing 
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his/her research. The attitude could well be that "the information is there 
for anyone who wants it." While this may be the approach of an 
individual researcher, it hardly seems an acceptable stance for someone 
who is a professional counselor to take. The continuing personal and 
professional renewal of each counselor is the concern of all of us for we 
are judged as a profession as much by the behavior of the worst as by the 
behavior of the best of us. 

As Jon Carlson noted in his quote from Chapter 6, the greatest 
challenge we face may be whether we have results that have the 
potential to significantly change counseling for the better, results which 
counselors will accept and then use to change their counseling behavior 
if it has become contrary to their current beliefs and actions. Do we 
selectively accept and use counseling research to buttress what we do 
and ignore that which challenges our present practices? To what purpose 
do we do research other than for personal ego gratification if it is 
without the potency to promote counselor change? 

We believe how research is framed and the involvement of 
stakeholders in critical decision points in the research and dissemination 
process will clearly influence how they respond to and use research 
results. But there is a step beyond that. It relates to the commitment each 
individual counselor has to her/his professional growth and renewal. We 
expect that commitment may well be built during the initial counselor 
education preparation. If a counselor is imbued with the joy of discovery 
and as an integral part of her/his learning is assisted in assimilating and 
acting on new information, even when it challenges long-held beliefs 
garnered from earlier course work and experience, the likelihood of 
movement or change occurring is enhanced. 

In addition to creating a climate conducive to change, two special 
features are important. First, is helping each new counselor to see the 
importance of establishing for him or herself a system for personal and 
professional renewal. Current certification programs, with their built-in 
support for continuing education, are helpful in this respect. 
Additionally, it is important that each person be prepared to assimilate 
and use information from a variety of information sources, e.g., journals, 
databases, specialized searches, and workshops and conferences. The 
important part here is having a systematic plan. There is a vast 
difference in outcome and level of learning between a person who just 
goes to a conference because it is the thing to do and a person who 
attends a conference seeking answers to specific questions and has 
thought through who she/he wishes to connect with while at the 
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conference. The former person leaves to chance whether and how the 
conference will impact upon him/her. The latter person knows what 
he/she wants to learn and has a plan for achieving it. The second feature 
of the renewing professional is an active networking that provides 
regular input from professional peers who function as a reliable source 
of interchange on important professional topics. New research results 
and considerations as to how the research can be used are particularly 
apt topics for interchange in the network. A stimulating new feature is 
the introduction of computer networking and conferencing where 
persons without respect to geography and time can b e active 
participants in discussions of implications and applications of a new 
piece of research. 

9. We need to be sensitive to the importance of the use of research 
methodologies and instrumentation that are appropriate for use with 
minorities. A major theme throughout this volume has been the 
importance of stakeholder involvement in the research process. In 
principle number 3 we emphasized the importance of "front-end framing 
which reflects significant stakeholder needs and interests..." This seems 
to us a cardinal principle of research if it is to be effectively 
disseminated and used to bring about meaningful change. It assumes 
special importance when the stakeholders are respectful of particular 
cultural, racial or gender groups. Without adequate representation 
subgroups can seriously challenge the relevance of the research 
outcomes. In recent discussions we have heard persons speaking from a 
feminist perspective seriously question the outcomes of many counseling 
studies because the design was "rigidly experimental" which they 
viewed as insensitive to gender differences. In their view, a qualitative 
design would have been more responsive to gender differences and led 
to different results. 

It is beyond the focus of this article to probe the merits of different 
research designs. However, from a dissemination and utilization 
standpoint there is a clear need for the viewpoints of any significant 
subpopulation to be acknowledged and represented in the design phase 
so that when the outcomes are disseminated the discussion is focused on 
how the outcomes can be utilized by different subgroups rather than 
incite needless haggling over the validity and utility of the results. Self- 
interest should clearly motivate researchers to be cognizant of the 
possible/probable responses of significant subpopulations to the research 
design and instrumentation phases of counseling outcomes research. By 
properly attending to subgroup's concerns in the front-end framing, an 
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outcome researcher enhances the probabilities that her/his research will 
be disseminated and that it will lead to meaningful change. And what 
else do we (or should we) do outcome research for other than to promote 
change? 

An interesting side note is a concern rightfully raised by Watkins and 
Schneider (1991) about the proliferation of outcome measures, many of 
them unstandardized. They suggest that "like counseling itself, 
counseling-outcome measurement sometimes appears to be in a state of 
chaos/* (p. 71). From the period 1983-1989 they counted 1430 outcomes 
measures in their review (840 of them being different ones). Obviously 
not a desirable state. It does seem that some attention to stakeholder 
interests in the front-end framing would lead to a greater degree of 
consensual approval about the suitability of different measures in 
responding to important subgroup characteristics. Given a known 
"approval rating" for different subgroup appropriateness for different 
instruments, this could lead to a greater use and continued refinement of 
such instruments rather than continuing the professionally unsound 
practice of the development of new, essentially unvalidated 
instrumentation for each new study. 

70. We need to use dissemination modes for reporting outcome 
research findings that are more broadly responsive to the needs and 
interests of all users. For most of us in our professional lives our reading 
time shrinks while our reading list grows. We keep hoping for a few 
"rainy days" where we can get to the piles of reading materials we have 
set aside for later. My recall of the studies on scientific communication 
conducted in the 1960s is that few persons in most professions read any 
particular article within a year of its publication. Though I lack 
information of more recent vintage, I believe that the earlier conclusions 
are still justified today. Though counseling was not investigated per se, I 
expect that it is no different than other professional groups in the 
relatively low level of readership of the journal literature proportional to 
the membership of the total professional group. Most authors we expect 
have had the underwhelming experience of pouring their heart and soul 
into a piece of writing and expectantly awaiting an avalanche of 
responses which never materialize (newsletter articles may be a happy 
exception to this phenomenon, an outcome which we will revisit 
shortly). 

If the lack of readership of our best journal articles, not to mention 
the lack of assimilation of their ideas, is a valid observation of current 
practice, a compelling question for any researcher, particularly outcome 
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researchers (who implicitly — if not explicitly — are concerned with the 
improvement of practice) is what vehicle(s) can be used to better 
communicate with the populations who need and can use valid research 
outcome findings? The question may be even more daunting than it first 
appears to be when one considers that, though ostensibly researchers are 
concerned with change and the improvement of practice, we can 
conclude from how they respond to the challenge that other variables 
such as the academic reward system are important considerations in 
where and how research results are reported. Such behavior should not 
be the source of blame. Probably most, if not all, of us are interested in 
peer judgments of our merits and promotability and only the most 
sophisticated faculty "merit and promotion" committees seek methods 
other than peer testimonials to assess the value and utility of a given 
person's research and writing. 

We believe, however, that increased attention and emphasis should be 
given to using new forms of disseminating research outcomes that more 
fully respond to the needs of today's counselors. 

Use of Meta-Analyses-Type Syntheses to Digest Large 
Bodies of Research Findings 

There are probably many more persons who would like to regularly read 
research articles in relevant journals than there are those who can find 
the time to do so, The meta-analyses type of review can periodically 
jumpstart a person's interest in and knowledge about a topic of 
importance to his/her counseling behavior. I use the term meta-analysis- 
type syntheses to designate a body of dissemination formats from which 
an author can select the format most appropriate to the research content 
and the intended audience. For some situations a formal, traditional 
meta-analysis is a best choice while in other situations a revised 
synthesis format is preferable. Many useful examples exist of traditional 
meta-analyses and research reviews of the AERA type of Annual Review 
of Research. We believe a recent article in Counselor Education and 
Supervision by Sexton and Wiston (1991) is an excellent example of a 
synthesis type of article which is a model of effective dissemination to a 
defined user population. In a relatively short space it reviews and 
synthesizes the empirical literature on three components of counseling, 
succinctly summarizes the relevant findings regarding each of these 
components, and concludes with a statement of the implications of their 
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synthesis for the research on training and supervision. The conclusion 
includes both a highlighting of the judgments of others as well as the 
authors' own interpretive comments regarding training needs and 
priorities. For a short expenditure of time, readers responding to this 
format can upgrade their knowledge about important counseling topics; 
identify specific studies to pursue in detail; and review a series of 
compelling and challenging conclusions to compare with their own 
thinking. Most importantly, it stimulates further thinking (and action!) 
rather than stifling it. 

The electronic revolution is promoting changes in professional 
communication which may go unnoticed by authors/researchers (and 
which may handicap them in effectively communicating in an electronic 
age). Two very important characteristics are (1) the need for data 
compression and (2) the explicit statement of findings and conclusions. 
Time is of the essence in electronic communication, hence lengthy 
documents (full text) are a rare exception and there is a continuing press 
to shorten and synthesize large documents into short, succinct ones. 
Users of electronic data systems also are typically operating from a 
sense of urgency and have a strongly felt need for rapid retrieval of 
important findings and author conclusions. 

There is, in fact, a third characteristic: An insatiable thirst for hot 
versus cold data— research findings that are relevant because the time 
lag between when the data were gathered, and when it becomes available 
was small enough so that the data could be generalized to larger 
populations. A common lag of 2 to 3 years between when the data are 
collected and a report is written and published, and an end user is able to 
obtain it, may preclude the data having real utility in a world where 
events change so rapidly. The need, therefore, is clearly for dis- 
semination formats that respond to at least the three specified criteria: 
(1) data compression, (2) explicit statement of findings, and (3) 
minimized reporting time lag. 



Digests and Abstracts 



Digests 

Probably the most currently sought after and extensively used 
dissemination mode is the digest. They are responsive to the three 
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previously identified criteria desirable for effective dissemination. 
Though varying in particulars, depending upon the sponsoring agency, 
digests have become increasingly attractive because of their relative ease 
of electronic communication and highly positive user response. The 
most comprehensive system for producing digests, and probably most 
extensively used digests, are those developed by the Educational 
Resources Information Center (ERIC) of the Office of Educational 
Research and Improvement (OERI). Designed to provide a synthesis of a 
relatively large body of information in a single, double-sided page (1200 
words), digests have earned the reputation for offering authoritative 
information and analyses on critical topics in education in a timely 
manner. Their popularity has been enhanced because they are available 
on-line — ERIC Digest on-line EDO — and are accessible through 
Internet or GTE. The ease of access and their public domain status has 
led to literally thousands of them being copied and distributed both in 
the US and abroad. ERIC/CAPS has developed two spinoffs of the 
digest system by offering a desktop compendium of digests relevant to 
counseling drawn from the digest collections of all ERIC Clearinghouses 
and placed in an easy-to-use volume of 168 digests entitled 
Counselor Quest. To offer information on fast-breaking educational 
developments which occur between volumes of CounselorQuest, a 
Counselor Quest Update Pack has been introduced which includes 
digests that arc literally only days old. Future innovations include the 
availability of packets which provide indepth coverage of a single topic 
(e.g., all digests in the system on career development). The low cost, 
popular acceptance, and varied uses to which they can be put, have led 
digests to become "best sellers" in information dissemination programs. 

Abstracts 

Embodying some of the same attractive features as digests, abstracts 
have become an indispensable tool in the retrieval of information from 
large data bases. Topically, abstracts in Resources in Education (RIE) 
run 200 words or less. They have proven to be extremely useful in aiding 
a user in deciding whether to retrieve a particular document or not. Also, 
reviewing all of the abstracts on a particular topic provides a quick, but 
remarkably cogent review of the major findings in the larger overall 
literature on a given topic. Though not as extensive, similar information 
advantages are offered by the shorter abstracts provided by 
Psychological Abstracts and Cumulative Index to Journal Education 
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(CIJE). Analysis of abstracts by frequency and type of topical coverage 
also provides useful trends and time analyses. As described earlier, a 
person who systematically reviews the abstracts of a given topic, e. g., 
school achievement studies, can keep him/herself on the leading edge of 
new information on topics of high personal interest, thus enriching the 
teaching and research capabilities of the person. 

Summary 

The central messages in this chapter can be subsumed under main points. 
First, the importance of front-end framing or the inclusion of the 
intended users of the research outcomes in the planning of the research, 
particularly with regard to the questions to be answered. Second, the 
need to tailor the dissemination mode to the users to insure a better fit 
between the informational content and a form which is conducive to the 
use of it by a counselor to bring about change. Third, the desirability of 
authors employing multiple dissemination modes more in tune with the 
electronic age (such as digests). Fourth, and pertiaps most important of 
all, thinking of dissemination as an interactive process between the 
researcher and the user with each contributing significantly to the 
process. 
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Chapter 9 
A Summing Up 

Garry R. Walz and Jeanne C. Bleuer 



Propositions Regarding Counselor Efficacy 

It is possible, even probable, that as you read the preceding chapters 
you were generating in your own mind thoughts about what 
contributes to counselor efficacy If you are like us, the visual path you 
developed is a diversified one — "well trodden" in places based on 
previous personal practices, in others only a visual image, and a belief 
based on your reading that a particular practice would be the way to go. 
We believe that is a desirable outcome — reinforcement of some current 
behaviors and a compelling vision of what can be done to enhance 
counselor efficacy. 

In this chapter, we offer a personal summing up of potent factors in 
counselor efficacy. Some of these factors are ones we have believed in 
for many years — others were formed as a result of our reading and 
writing the previous chapters. What we believe is of major importance is 
the stimulus value that the writing has to facilitate examination of 
personal belief systems and professional practices. Increased motivation 
to examine one's counseling beliefs and practices may be the really 
significant long term outcome from reading (and hopefully reflecting on) 
the contents of this volume. 
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It has been said that one experiences the really momentous events in 
life alone. No matter how close a person may be to another, the actual 
experience of a life event, fleeting or prolonged, is experienced 
singularly — by oneself. How we experience and relate to any given 
event is unique. We believe that many of the factors discussed in this 
volume have meaning to all counselors but are mediated through their 
personal belief systems and life experiences. We believe this personal 
mediation makes the development of counseling axioms contraindicated. 
Rather, we see each person making her/his own set of efficacy 
propositions as a means to harness and target personal energy. The result 
is a dynamic and life long personal and professional renewal, focused 
upon enhancing a person's (a counselor's) efficacy. 

Though we believe each person experiences significant life events 
essentially alone, we also are of the view that discourse on personal and 
professional beliefs is rewarding. Whether confirmatory or challenging, 
it serves to strengthen and/or broaden and change personal beliefs. We 
therefore offer these propositions as a stimulus to others in undertaking a 
rigorous and spirited analysis of counselor efficacy — their own and that 
which applies more broadly to the profession. 

Proposition 1 — Counseling does make a difference* 

Though the exact effects of counseling vary with the person and 
problem, the evidence is both so strong and so consistent across problem 
and client types that a given counselor can operate with the belief that 
her/his efforts will make a difference. We believe that this research 
knowledge buttresses a counselor's personal beliefs based on his/her own 
experiences. We believe having a firm conviction that counseling is a 
powerful tool for positive change in human behavior is important. It 
reinforces the belief of counselors in the efficacy of their efforts. Thus, 
they are likely to perform with greater confidence and believe more 
strongly in the worth of what they do. 

We sense many counselors act in a defensive, even apologetic 
manner, especially in dealing with the public at large. The data-based 
substantiation provided in this volume should spur counselors on to 
further develop the efficacy of their counseling. In particular, it will 
support conceptual and practical excursions into new counseling 
domains that counselors less sure of their efficacy would shy away from. 
Here we are thinking of a need for greater responsiveness to 
multiculturalism and abuse, as well as to new tools such as computer- 
assisted counseling. 
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Proposition 2 — Counselor efficacy can be enhanced through regular 
dialogue with peers regarding significant counseling developments 
using electronic networking. 

We believe that counseling beliefs are a product of: 1) a counselor's 
counseling experiences, 2) what they read, and 3) what they learn about 
counseling from interaction with knowledgeable peers (whose beliefs are 
strongly influenced by 1 & 2). Numerous counselors have shared with us 
that since leaving their counselor education programs they have had few 
real opportunities to discuss, either in depth, or over an extended period 
of time, questions about or views regarding counseling. This opportunity 
to discuss significant counseling topics with peers is one of the things 
they miss most about their days training to become counselors. It also is 
one of the aspects of conferences and conventions which they appreciate 
most — discussing their counseling experiences with peers. The 
discussions that go on while "conventioneering," however, frequently 
focus as much or more on topics of governance and broad professional 
issues as they do specific counseling topics. 

Recent years have witnessed enormous advances in the quality and 
availability of computer networking. Internet in particular, a 
supernetwork of computer networks, is a means for rapid 
communication between people. As many as 10 million people may now 
be served by Internet around the world. The technological capability is 
also available through special networks for computer conferencing. It 
would seem that the possibilities for highly useful and rewarding 
communication for counselors now readily exists. Such communication 
is now a regular part of the professional lives of many educational 
groups (e.g., librarians, college professors) and individual counselors. 
Special counselor computer conferences and networks dedicated to 
counseling topics and composed of participants with common interests 
offer great promise to counselors. Topics might include best practices in 
career counseling, new counseling interventions which have worked, or 
counseling research outcomes with relevance for career counselors, 
college counseling center counselors, and counselor educators. These 
conferences and networking opportunities would both speed the 
dissemination of new ideas and research results and encourage 
interaction between participants. 

Participation in such electronic conferences involving a regular bi- 
weekly discussion offers much in broadening and strengthening a 
counselor's efficacy. In this case, since Internet and comparable 
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networks are so relatively inexpensive, the will to do it seems more 
important than the wherewithal to support it. 

Proposition 3— The systematic acquisition and utilization of research 
results must become a mainstay in the professional counseling 
behavior of counselor educators and counselors. 

An area that gives us pause in reflecting on counselor efficacy is the 
entire research enterprise as it relates to counseling. There are many 
apparent disparities. Counseling is seen as important yet knowledge of 
counseling research is limited. Many counselors apparently ignore 
information which conflicts with their prevailing views. Overall, many 
counselors suggest research results do not play a very important role in 
the development of their counseling approaches. While there are 
undoubtedly many explanations for this state of mind, we would suggest 
one explanation is certainly the low level of emphasis given to the 
acquisition and utilizction of research results. 

It is likely that relatively few persons will conduct research studies, 
but all counselors can and should be involved in incorporating new 
research results into their thinking and their behavior. Perhaps we need 
to give greater attention during in-service educational activities for 
counselors to how to incorporate new ideas into existing belief systems. 
Particularly helpful would be a consumer approach to research results 
where counselors were helped to consider how research results should be 
examined and to attend to how a research finding would be translated 
into particular emphases and behaviors on the part of a counselor. Most 
important of all is the need to place greater emphasis on change in 
counseling behavior being data driven rather than something that 
happens by whim or fancy 

Proposition 4— Research User Participation in the Framing of 
Research Designs and Programs Will Lead to Greater Appreciation 
and Use of Counseling Research. 

The low repute in which counseling research is held by many 
counselors has been a concern of many of the writers in this volume. On 
the one hand we have a growing body of research results which validate 
the counseling enterprise and assist in differentiating between desirable 
and less desirable counseling practices. A clear plus for counseling. On 
the other hand, we have large numbers of counselors (larger by far than 
the first group) who regard counseling research as essentially worthless. 
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They see few, if any, applications of the research to the problems and 
issues which they face. 

Admittedly, the problem is a difficult one and not easily resolved. It 
is long standing and represents differences in priorities and perspectives. 
It would seem, however, that a greater use of research driven change in 
counseling could be accomplished by giving persons who are the 
intended users and beneficiaries of the research the opportunity to design 
and contribute to the research. For many counselors counseling research 
is seen as "worthless" because it doesn't attend to the needs and interests 
of counseling practitioners. Broader participation in the framing of 
research will likely result in a greater use of the results — a highly desired 
outcome of the research. 

Proposition 5 — A "Common Factors" Approach to What Makes for 
Successful Counseling Offers Considerable Promise for Promoting 
Enhanced Counselor Efficacy. 

We believe the discussion by Sexton in Chapter 7 of the common 
factors across different successful approaches to counseling would do 
much to develop a higher level integration of counseling approaches that 
would greatly benefit both counseling practitioners and researchers. It is 
a functional way of skimming the cream from a large number of 
approaches, none of which has been able to demonstrate any clear 
superiority over the others, but each of which has something to 
contribute to a new, more pervasive and inclusive paradigm of effective 
counseling interventions. It has the further advantage of providing the 
opportunity for contributions to be drawn from diverse sources. By 
broadening the base of experience from which the basic tenets are 
drawn, the probability of its applicability to a wider range of counseling 
clients and situations is increased. We believe this endeavor would be a 
useful one in bringing together and synthesizing the best of our 
counseling research and practice results. It would further create a 
paradigm for counseling interventions that counselors would find more 
useful and one that would be inclusive of their own personal experiences 
and thinking. Far better to operate from this inclusive and synthesizing 
approach than to cany on in "horse racing" fashion pitting one approach 
against another to determine the "winner." 

Proposition 6 — The capacity to understand multiculturalism, to see 
the strength in diversity, and to possess the desire and the ability to 
create community are essential variables in counselor efficacy. 
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At whatever level a counselor is at or in what particular situation they 
function as a counselor, their understanding of the changing milieu in 
which they function is critical to their effectiveness. Counselors need to 
be able to understand the importance of the effect of racial/ethnic 
background on a person's socialization while also recognizing the 
individuality and uniqueness of each person. Of great importance is that 
counselors, both in attitude and behavior, communicate their belief in the 
strength that diversity and pluralism bring to an organization. The 
counselor must not play a passive role, but rather an active one of 
building community where people lower their defenses and rejoice in the 
differences among people. Counselors are in a unique position to see the 
pervasive negative effects of a lack of community where the "in" culture 
are viewed as inferior. The counselor's ethical and professional 
obligation is to assist each client to cope with the culture as they 
experience it. But they also have an unusual opportunity to assist the 
community in liberating itself from dysfunctional attitudes and behaviors 
which impact on individuals in ways that deprive them of their right to 
be different and to be seen by themselves and others as worthy persons. 



Conclusion 

Our review and analysis has led us to believe that counselor efficacy is a 
worthy topic of consideration for all counselors— irrespective of their 
position and perspective. It is a process of sifting and sorting from all the 
data— both experiential and empirical— that which fits a particular 
counselor from that which does not. Counselor efficacy has both 
external and internal referents— the knowledge a counselor acts upon 
must be shown to be worthy (external), and each counselor must realize 
that it has meaning and utility for him/herself (internal). Counseling 
efficacy will have different definitions for different counselors. 

We hope, however, that the ideas discussed herein will become a road 
not less, but more frequently, travelled that leads to enhanced counselor 
effectiveness and personal satisfaction. 
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Educational Resources Information Center — ERIC 

ERIC is a decentralized nationwide information system founded in 1966 
and currently sponsored by the Office of Educational Research and 
Improvement within the U.S. Department of Education. It is the largest 
education related database in the world. ERIC is designed to collect 
educational documents and journal articles and to make them readily 
available through a number of products and services: the ERIC database, 
abstract journals, microfiche collections, online and CD-ROM computer 
searches, document reproductions, and information analysis publica- 
tions. The ERIC audience is equally wide-ranging and includes teachers, 
counselors, administrators, supervisors, policy makers, librarians, media 
specialists, researchers, students, parents, and other educators and 
interested persons. 

Counseling and Personnel Services 
Clearinghouse— CAPS 

CAPS is one of the 16 subject -oriented clearinghouses of the ERIC 
system. CAPS' exceptionally broad coverage includes K-12 counseling 
and guidance, post-secondary and adult counseling services, and human 
resource development in business, industry and government. Topics 
addressed by CAPS are many and varied and include: preparation, 
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practice and supervision of counseling professionals; career planning 
and development; marriage and family counseling; and services to 
special populations (substance abusers, students at-risk). 
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Descriptions 



Crfcme de la Crfeme Series: 

Empowering Young Black Males by Courtland C. Lee 

Focuses on the empowerment model Dr. Lee has 
developed for assisting young Black males in acquiring 
skills which will enable them to cope effectively. 
Includes four training modules and useful resources. 
95 pp. 

Learning Styles Counseling by Shirley A. Griggs 

Prepares counselors K-12 to diagnose and infuse 
learning style approaches into their counseling. Useful 
in consulting with teachers on accommodating learning 
styles in the classroom. 161 pp. 
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The Teacher Advisor Program: An Innovative 
Approach to School Guidance by Robert D. Myrick 
and Linda S. Myrick 

Describes six teacher advisor programs that creatively 
meet the developmental needs of middle and high 
school students. Many practical ideas and approaches. 
125 pp. 

Invitational Learning for Counseling and Develop- 
ment by William W. Purkey and John J. Schmidt 

Explains how integrating the principles of invitational 
learning into guidance was a positive force for change 
in seven schools. An excellent approach for responding 
to students at risk and minimizing dropouts. 138 pp. 

Comprehensive Guidance Programs that Work by 
Norman C. Gysbers 

Describes successful comprehensive guidance 
programs in eight school settings in six states. Includes 
practical suggestions for bringing about change in 
K-12 school guidance programs to make them truly 
comprehensive. 171 pp. 



School Counseling Series (ASCA and ERIC/CAPS): 

Toward the Transformation of Secondary School 
Counseling by Doris Rhea Coy, Claire Cole, Wayne C. 
Huey and Susan Jones Sears, Editors 

This compilation of original articles and the best from 
counseling journals provides a discussion of key 
concerns, issues and trends in secondary school 
counseling. Developmental guidance programs, teenage 
fathers, computer competency, the counselor's role in 
enhancing the school climate, and using career 
information with dropouts and at-risk students are some 
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of the topics included in the eight chapters and 35 
articles. 393 pp. 

The Challenge of Counseling in Middle Schools by 
Edwin R. Gerler, Jr., Canary C. Hogan and Kathleen 
O'Rourke, Editors 

A compilation of articles from journals and ERIC, as 
well as one original article, on topics of importance to 
middle school counselors. 386 pp. 

Elementary School Counseling in a Changing World 
by Edwin R. Gerler, Jr., Joseph C. Ciechalsld and Larry 
E. Parker, Editors 

This book is a collection of the best articles published 
in counseling journals during the 1980s as well as 
original articles, providing elementary specialists with 
an abundance of eye-opening, innovative approaches 
and useful information as they seek the surest methods 
for guiding young students. 398 pp. 



Other Exciting CAPS Publications: 

Family Counseling and Therapy by Robert L. Smith 
and Patricia Stevens-Smith 

Provides an interesting and incisive overview of 
marriage and family counseling. Six chapters provide 
extensive coverage of: Healthy Family Functioning; 
Marriage and Family Counseling Theories; the Practice 
of Marriage and Family Counseling; Training Marriage 
and Family Counselors/Therapists; Issues and Topics in 
Family Therapy; and The Future — Images and 
Projections. An exceptional blend of previously printed 
and original articles by an array of eminent authors. 
Includes a stimulating introduction by Samuel 
Gladding. An excellent classroom text or resource for 
counselor updating. 446 pp. 
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Counselor-Quest Compiled by Garry R. Walz 

CounselorQuest is unique among publications in the 
amount of practical and reliable information it offers 
counselors on a wide variety of topics of critical impor- 
tance to them. Over 165 succinct and highly readable 
digests and three indexes (title, topic, and educational 
level). Each digest offers a comprehensive overview of 
a topic, provides useful practice and program sugges- 
tions, and tells you where to go for more information. 
An indispensable counseling tool that will save you 
from hours of searching for the information you need. 
Reproducible for use in classes, counseling interviews, 
and group meetings, too! 350 pp. 

Counseling Underachieves by Jeanne C. Bleuer 

Summarizes achievement theories, shedding new light 
on the interaction between ability and effort. Presents a 
comprehensive model for counseling interventions. 
Includes guidelines for identifying the factors involved 
in student underachievement, and practical materials 
and ideas for improving study skills. 92 pp. 

Helping Children Cope With Fears and Stress by 
Edward H. Robinson, Joseph C. Rotter, Mary Ann Fey, 
and Kenneth R. Vogel 

A concise overview of children's fears and stress. 
Features 47 activities and strategies for individual and 
group counseling interventions or for easy integration 
into the K-12 curriculum. Includes a facilitator's guide 
for training teachers and counselors on how to help 
children cope with fears and stress. 178 pp. 

Developing Support Groups for Students: Helping 
Students Cope With Stress by Garry R. Walz and 
Jeanne C. Bleuer, Editors 
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This volume includes six information-packed modules 
offering clear instructions on how to utilize the proven 
power of student support groups to assist students in 
developing well-balanced and emotionally stable 
personalities. Modules include: Helping Students Cope 
With Fears and Crises; Programs and Practices: 
Developing and Offering Student Self-Help Support 
Groups: Designing and Implementing Student Support 
Programs; Abstracts of Significant Resources; Sources 
for Assistance and Consultation, 202 pp. 

Student Self-Esteem: A Vital Element of School 
Success by Garry R. Walz and Jeanne C Bleuer, 
Editors 

A comprehensive and highly useful resource on K-12 
student self-esteem and staff development for 
counselors, educators, and teachers. Sixteen chapters 
provide over 60 articles written by outstanding 
authorities on self-esteem, offering practical and field- 
validated program and practice ideas, 510 pp. (Jointly 
developed by ERIC/CAPS and ASCA) 

Counseling Young Students at Risk by Jeanne C. 
Bleuer and Penny A, Schreiber, Editors 

Examines family and environmental pressures which 
make school success very difficult for many of today's 
young students. Includes an extensive array of 
suggestions and resources on: latchkey children; 
children of divorce; children of alcoholic or abusive 
parents; stress management for children; helping 
children cope with death and loss; and the relationship 
between children's self-esteem and their performance in 
school. 750 pp. 

Counseling Futures by Garry R. Walz, Editor 

Features revised and expanded papers prepared by 
George Gazda, Bruce Shertzer, and Garry Walz 
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presented at the Chi Sigma Iota Symposium on the 
Future of Counseling. Offers a series of exciting and 
stimulating perspective on the future of counseling. 
86 pp. 

Empowerment for Later Life by Jane E. Myers 

Examines the effects of aging on self-esteem and 
empowerment and reviews developmental issues key to 
understanding the aging process. Offers a "holistic 
wellness model" for empowering older persons. 118 pp. 
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Few questions are as uppermost in the thinking of counselors as 
what contributes to counseling and counselor efficacy. Does coun- 
seling make a difference? What counseling approaches are the most 
effective, the least? What can be said about counseling efficacy in 
different specialties such as school, career, and marriage and family 
counseling? Is there a one hast approach to counseling? These and 
many other compelling issues and questions are addressed in this 
unique book. Eight eminent writers and counseling leaders have 
reviewed the literature and prepared cogent analyses. Included in 
each chapter Is a thorough review of the literature as well as 
thoughtful and highly useful recommendations. Launched as an 
ERIC/CAPS and ACES project, this has grown to become a volume 
filled with tarights and ideas that are useful to counseling practi- 
tioners as well as counseling educators. Among the exciting chap- 
ters are: * - 
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